2021-09-10 16:46:20.712 4076 3050945804776 MainThread DEBUG os_vif [-] Loaded VIF plugin class '' with name 'linux_bridge' initialize c:\python37\lib\site-packages\os_vif\__init__.py:46 2021-09-10 16:46:20.712 4076 3050945804776 MainThread DEBUG os_vif [-] Loaded VIF plugin class '' with name 'noop' initialize c:\python37\lib\site-packages\os_vif\__init__.py:46 2021-09-10 16:46:20.712 4076 3050945804776 MainThread DEBUG os_vif [-] Loaded VIF plugin class '' with name 'ovs' initialize c:\python37\lib\site-packages\os_vif\__init__.py:46 2021-09-10 16:46:20.737 4076 3050945804776 MainThread INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs 2021-09-10 16:46:22.077 4076 3050945804776 MainThread INFO nova.virt.driver [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Loading compute driver 'hyperv.driver.HyperVDriver' 2021-09-10 16:46:22.948 4076 3050945804776 MainThread INFO nova.compute.provider_config [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access. 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_concurrency.lockutils [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Acquired lock "singleton_lock" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_concurrency.lockutils [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Releasing lock "singleton_lock" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Full set of CONF: _wait_for_exit_or_signal c:\python37\lib\site-packages\oslo_service\service.py:362 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ******************************************************************************** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2589 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Configuration options gathered from: log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2590 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] command line args: ['--config-file', 'c:\\openstack\\etc\\nova.conf'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2591 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] config files: ['c:\\openstack\\etc\\nova.conf'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2593 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ================================================================================ log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2594 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] allow_resize_to_same_host = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] arq_binding_timeout = 300 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] backdoor_port = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] backdoor_socket = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] block_device_allocate_retries = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] block_device_allocate_retries_interval = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cert = self.pem log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:22.998 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute_driver = hyperv.driver.HyperVDriver log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute_monitors = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] config_dir = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] config_drive_format = iso9660 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] config_file = ['c:\\openstack\\etc\\nova.conf'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] config_source = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] console_host = n-h1-808474-1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] control_exchange = nova log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cpu_allocation_ratio = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] daemon = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] debug = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] default_access_ip_network_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] default_availability_zone = nova log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.014 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] default_ephemeral_format = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] default_schedule_zone = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] disk_allocation_ratio = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] enable_new_services = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] enabled_apis = ['osapi_compute', 'metadata'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] enabled_ssl_apis = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] flat_injected = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] force_config_drive = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] force_raw_images = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] graceful_shutdown_timeout = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] heal_instance_info_cache_interval = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] host = n-h1-808474-1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] initial_cpu_allocation_ratio = 16.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] initial_disk_allocation_ratio = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] initial_ram_allocation_ratio = 1.5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] injected_network_template = c:\openstack\build\nova\nova/virt/interfaces.template log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_build_timeout = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_delete_interval = 300 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_format = [instance: %(uuid)s] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_name_template = instance-%08x log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_usage_audit = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_usage_audit_period = hour log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instance_uuid_format = [instance: %(uuid)s] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] instances_path = C:\openstack\instances log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] internal_service_availability_zone = internal log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] key = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] live_migration_retry_count = 30 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_config_append = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_date_format = %Y-%m-%d %H:%M:%S log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_dir = c:\openstack\log log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_file = nova-compute.log log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_options = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_rotate_interval = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.022 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_rotate_interval_type = days log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] log_rotation_type = none log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(thread)d %(threadName)s %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] logging_debug_format_suffix = %(funcName)s %(pathname)s:%(lineno)d log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] logging_default_format_string = %(asctime)s.%(msecs)03d %(process)d %(thread)d %(threadName)s %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] logging_exception_prefix = %(asctime)s.%(msecs)03d %(process)d %(thread)d %(threadName)s TRACE %(name)s %(instance)s log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] long_rpc_timeout = 1800 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_concurrent_builds = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_concurrent_live_migrations = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_concurrent_snapshots = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_local_block_devices = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_logfile_count = 30 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] max_logfile_size_mb = 200 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] maximum_instance_delete_attempts = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metadata_listen = 0.0.0.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metadata_listen_port = 8775 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metadata_workers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] migrate_max_retries = -1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] mkisofs_cmd = c:\openstack\bin\mkisofs.exe log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] my_block_storage_ip = 10.222.0.32 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] my_ip = 10.222.0.32 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] network_allocate_retries = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.046 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] osapi_compute_listen = 0.0.0.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] osapi_compute_listen_port = 8774 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] osapi_compute_unique_server_name_scope = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] osapi_compute_workers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] password_length = 12 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] periodic_enable = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] periodic_fuzzy_delay = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] pointer_model = usbtablet log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] preallocate_images = none log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] publish_errors = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] pybasedir = c:\openstack\build\nova log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ram_allocation_ratio = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rate_limit_burst = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rate_limit_except_level = CRITICAL log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rate_limit_interval = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reboot_timeout = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reclaim_instance_interval = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] record = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] report_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rescue_timeout = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reserved_host_cpus = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reserved_host_disk_mb = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reserved_host_memory_mb = 512 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] reserved_huge_pages = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] resize_confirm_window = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.061 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] resize_fs_using_block_device = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] resume_guests_state_on_host_boot = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rootwrap_config = /etc/nova/rootwrap.conf log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rpc_response_timeout = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] run_external_periodic_tasks = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] running_deleted_instance_action = reap log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] running_deleted_instance_poll_interval = 120 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] running_deleted_instance_timeout = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler_instance_sync_interval = 120 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_down_time = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] servicegroup_driver = db log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] shelved_offload_time = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] shelved_poll_interval = 3600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] shutdown_timeout = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] source_is_ipv6 = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ssl_only = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] state_path = c:\openstack\build\nova log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] sync_power_state_interval = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] sync_power_state_pool_size = 1000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] syslog_log_facility = LOG_USER log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] tempdir = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] timeout_nbd = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] transport_url = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] update_resources_interval = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.076 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_cow_images = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_eventlog = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_journal = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_json = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_rootwrap_daemon = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_stderr = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] use_syslog = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vcpu_pin_set = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plugging_is_fatal = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plugging_timeout = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] virt_mkfs = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] volume_usage_poll_interval = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] watch_log_file = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] web = /usr/share/spice-html5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2603 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_concurrency.lock_path = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_metrics.metrics_buffer_size = 1000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_metrics.metrics_enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_metrics.metrics_process_name = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.auth_strategy = keystone log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.compute_link_prefix = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.093 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.dhcp_domain = novalocal log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.enable_instance_password = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.glance_link_prefix = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.instance_list_cells_batch_fixed_size = 100 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.instance_list_cells_batch_strategy = distributed log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.instance_list_per_project_cells = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.list_records_by_skipping_down_cells = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.local_metadata_per_cell = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.max_limit = 1000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.metadata_cache_expiration = 15 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.108 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.neutron_default_tenant_id = default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.use_forwarded_for = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.use_neutron_default_nets = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_dynamic_connect_timeout = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_dynamic_failure_fatal = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_dynamic_read_timeout = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_dynamic_ssl_certfile = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_dynamic_targets = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_jsonfile_path = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api.vendordata_providers = ['StaticJSON'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.backend = dogpile.cache.null log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.backend_argument = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.config_prefix = cache.oslo log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.debug_cache_backend = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.expiration_time = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_dead_retry = 300 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_pool_connection_get_timeout = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_pool_flush_on_reconnect = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_pool_maxsize = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_pool_unused_timeout = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_servers = ['localhost:11211'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.memcache_socket_timeout = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.proxies = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.tls_allowed_ciphers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.tls_cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.tls_certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.tls_enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cache.tls_keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.auth_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.catalog_info = volumev3::publicURL log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.cross_az_attach = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.116 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.endpoint_template = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.http_retries = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.os_region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cinder.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.consecutive_build_service_disable_threshold = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.cpu_dedicated_set = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.cpu_shared_set = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.image_type_exclude_list = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.live_migration_wait_for_vif_plug = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.max_concurrent_disk_ops = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.max_disk_devices_to_attach = -1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.provider_config_location = /etc/nova/provider_config/ log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.resource_provider_association_refresh = 300 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] compute.shutdown_retry_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] conductor.workers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] console.allowed_origins = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] console.ssl_ciphers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] console.ssl_minimum_version = default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] consoleauth.token_ttl = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.139 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.service_type = accelerator log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] cyborg.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.backend = sqlalchemy log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.connection = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.connection_debug = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.connection_parameters = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.connection_recycle_time = 3600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.155 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.connection_trace = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.db_inc_retry_interval = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.db_max_retries = 20 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.db_max_retry_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.db_retry_interval = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.max_overflow = 50 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.max_pool_size = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.max_retries = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.mysql_enable_ndb = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.mysql_sql_mode = TRADITIONAL log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.pool_timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.retry_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.slave_connection = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] database.sqlite_synchronous = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.backend = sqlalchemy log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.connection = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.connection_debug = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.connection_parameters = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.connection_recycle_time = 3600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.connection_trace = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.db_inc_retry_interval = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.db_max_retries = 20 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.db_max_retry_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.db_retry_interval = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.170 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.max_overflow = 50 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.max_pool_size = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.max_retries = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.mysql_enable_ndb = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.mysql_sql_mode = TRADITIONAL log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.pool_timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.retry_interval = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.slave_connection = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] api_database.sqlite_synchronous = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] devices.enabled_mdev_types = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ephemeral_storage_encryption.cipher = aes-xts-plain64 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ephemeral_storage_encryption.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ephemeral_storage_encryption.key_size = 512 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread WARNING oslo_config.cfg [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Deprecated: Option "api_servers" from group "glance" is deprecated for removal ( Support for image service configuration via standard keystoneauth1 Adapter options was added in the 17.0.0 Queens release. The api_servers option was retained temporarily to allow consumers time to cut over to a real load balancing solution. ). Its value may be silently ignored in the future. 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.api_servers = ['http://10.222.0.55/image'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.debug = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.default_trusted_certificate_ids = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.enable_certificate_validation = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.enable_rbd_download = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.186 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.num_retries = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.rbd_ceph_conf = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.rbd_connect_timeout = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.rbd_pool = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.rbd_user = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.service_type = image log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.verify_glance_signatures = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] glance.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] guestfs.debug = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.config_drive_cdrom = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.config_drive_inject_password = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.202 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.dynamic_memory_ratio = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.enable_instance_metrics_collection = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.enable_remotefx = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.instances_path_share = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.iscsi_initiator_list = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.limit_cpu_features = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.mounted_disk_query_retry_count = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.mounted_disk_query_retry_interval = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.power_state_check_timeframe = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.power_state_event_polling_interval = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.qemu_img_cmd = c:\openstack\bin\qemu-img.exe log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.use_multipath_io = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.volume_attach_retry_count = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.volume_attach_retry_interval = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.vswitch_name = br-data log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] hyperv.wait_soft_reboot_seconds = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] mks.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] mks.mksproxy_base_url = http://127.0.0.1:6090/ log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.manager_interval = 2400 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.precache_concurrency = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.remove_unused_base_images = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.remove_unused_original_minimum_age_seconds = 86400 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.218 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.remove_unused_resized_minimum_age_seconds = 3600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] image_cache.subdirectory_name = _base log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.api_max_retries = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.api_retry_interval = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.auth_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.partition_key = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.peer_list = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.serial_console_state_timeout = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.service_type = baremetal log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.233 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ironic.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] key_manager.backend = barbican log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] key_manager.fixed_key = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.auth_endpoint = http://localhost/identity/v3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.barbican_api_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.barbican_endpoint = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.barbican_endpoint_type = public log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.barbican_region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.number_of_retries = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.retry_delay = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.send_service_user_token = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.verify_ssl = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican.verify_ssl_path = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.248 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.auth_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] barbican_service_user.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.approle_role_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.approle_secret_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.kv_mountpoint = secret log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.kv_version = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.root_token_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.ssl_ca_crt_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.use_ssl = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vault.vault_url = http://127.0.0.1:8200 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.264 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.service_type = identity log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] keystone.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.connection_uri = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.cpu_mode = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.cpu_model_extra_flags = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.cpu_models = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.device_detach_attempts = 8 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.280 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.device_detach_timeout = 20 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.disk_cachemodes = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.disk_prefix = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.enabled_perf_events = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.file_backed_memory = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.gid_maps = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.hw_disk_discard = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.hw_machine_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_rbd_ceph_conf = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_rbd_glance_copy_poll_interval = 15 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_rbd_glance_copy_timeout = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_rbd_glance_store_name = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_rbd_pool = rbd log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_type = default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.images_volume_group = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.inject_key = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.inject_partition = -2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.inject_password = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.iscsi_iface = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.iser_use_multipath = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_bandwidth = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_completion_timeout = 800 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_downtime = 500 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.295 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_downtime_delay = 75 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_downtime_steps = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_inbound_addr = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_permit_auto_converge = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_permit_post_copy = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_scheme = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_timeout_action = abort log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_tunnelled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_uri = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.live_migration_with_native_tls = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.max_queues = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.mem_stats_period_seconds = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.nfs_mount_options = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.nfs_mount_point_base = c:\openstack\build\nova\mnt log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_aoe_discover_tries = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_iser_scan_tries = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_memory_encrypted_guests = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_nvme_discover_tries = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_pcie_ports = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.num_volume_scan_tries = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.pmem_namespaces = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.quobyte_client_cfg = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.quobyte_mount_point_base = c:\openstack\build\nova\mnt log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rbd_connect_timeout = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.311 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rbd_destroy_volume_retries = 12 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rbd_destroy_volume_retry_interval = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rbd_secret_uuid = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rbd_user = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.realtime_scheduler_priority = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.remote_filesystem_transport = ssh log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rescue_image_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rescue_kernel_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rescue_ramdisk_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rng_dev_path = /dev/urandom log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.rx_queue_size = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.smbfs_mount_options = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.smbfs_mount_point_base = c:\openstack\build\nova\mnt log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.snapshot_compression = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.snapshot_image_format = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.snapshots_directory = C:\openstack\instances/snapshots log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.sparse_logical_volumes = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.swtpm_enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.swtpm_group = tss log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.swtpm_user = tss log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.sysinfo_serial = unique log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.tx_queue_size = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.327 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.uid_maps = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.use_virtio_for_bridges = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.virt_type = kvm log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.volume_clear = zero log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.volume_clear_size = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.volume_use_multipath = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_cache_path = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_mount_group = qemu log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_mount_opts = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_mount_perms = 0770 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_mount_point_base = c:\openstack\build\nova\mnt log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.vzstorage_mount_user = stack log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] libvirt.wait_soft_reboot_seconds = 120 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread WARNING oslo_config.cfg [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Deprecated: Option "auth_plugin" from group "neutron" is deprecated. Use option "auth_type" from group "neutron". 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.auth_type = v3password log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.default_floating_pool = nova log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.343 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.extension_sync_interval = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.http_retries = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.metadata_proxy_shared_secret = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.ovs_bridge = br-int log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.physnets = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.region_name = RegionOne log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.service_metadata_proxy = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.service_type = network log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] neutron.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] notifications.bdms_in_notifications = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] notifications.default_level = INFO log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] notifications.notification_format = unversioned log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.358 4076 3050945804776 MainThread WARNING oslo_config.cfg [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] Deprecated: Option "notify_on_state_change" from group "DEFAULT" is deprecated. Use option "notify_on_state_change" from group "notifications". 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] notifications.notify_on_state_change = vm_and_task_state log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] notifications.versioned_notifications_topics = ['versioned_notifications'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] pci.alias = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] pci.passthrough_whitelist = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.auth_type = password log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.auth_url = http://10.222.0.55/identity/v3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.connect_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.connect_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.373 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.default_domain_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.default_domain_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.domain_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.domain_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.endpoint_override = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.max_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.min_version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.password = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.project_domain_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.project_domain_name = Default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.project_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.project_name = service log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.region_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.service_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.service_type = placement log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.status_code_retries = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.status_code_retry_delay = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.system_scope = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.trust_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.user_domain_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.user_domain_name = Default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.user_id = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.username = placement log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.valid_interfaces = ['internal', 'public'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] placement.version = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] powervm.disk_driver = localdisk log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] powervm.proc_units_factor = 0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] powervm.volume_group_name = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.cores = 20 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.count_usage_from_placement = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.383 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.driver = nova.quota.DbQuotaDriver log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.injected_file_content_bytes = 10240 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.injected_file_path_length = 255 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.injected_files = 5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.instances = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.key_pairs = 100 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.metadata_items = 128 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.ram = 51200 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.recheck_quota = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.server_group_members = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] quota.server_groups = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rdp.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.discover_hosts_in_cells_interval = -1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.enable_isolated_aggregate_filtering = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.image_metadata_prefilter = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.limit_tenants_to_placement_aggregate = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.max_attempts = 3 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.max_placement_results = 1000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.placement_aggregate_required_for_tenants = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.query_placement_for_availability_zone = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.405 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.query_placement_for_image_type_support = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.query_placement_for_routed_network_aggregates = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] scheduler.workers = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.aggregate_image_properties_isolation_namespace = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.aggregate_image_properties_isolation_separator = . log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.build_failure_weight_multiplier = 1000000.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.cpu_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.disk_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.host_subset_size = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.image_properties_default_architecture = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.io_ops_weight_multiplier = -1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.isolated_hosts = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.isolated_images = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.max_instances_per_host = 50 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.max_io_ops_per_host = 8 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.pci_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.ram_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.shuffle_best_same_weighed_hosts = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.soft_affinity_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.track_instance_changes = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.420 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metrics.required = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metrics.weight_multiplier = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metrics.weight_of_unavailable = -10000.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] metrics.weight_setting = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.base_url = ws://127.0.0.1:6083/ log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.port_range = 10000:20000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.proxyclient_address = 127.0.0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.serialproxy_host = 0.0.0.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] serial_console.serialproxy_port = 6083 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.auth_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.send_service_user_token = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] service_user.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.agent_enabled = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.436 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.html5proxy_host = 0.0.0.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.html5proxy_port = 6082 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.server_listen = 127.0.0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] spice.server_proxyclient_address = 127.0.0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] upgrade_levels.baseapi = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] upgrade_levels.cert = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] upgrade_levels.compute = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] upgrade_levels.conductor = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] upgrade_levels.scheduler = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.auth_section = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.auth_type = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.cafile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.certfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.collect_timing = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.keyfile = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.split_loggers = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vendordata_dynamic_auth.timeout = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.api_retry_count = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.ca_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.cache_prefix = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.cluster_name = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.connection_pool_size = 10 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.452 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.console_delay_seconds = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.datastore_regex = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.host_ip = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.host_password = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.host_port = 443 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.host_username = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.insecure = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.integration_bridge = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.maximum_objects = 100 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.pbm_default_policy = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.pbm_enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.pbm_wsdl_location = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.468 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.serial_log_dir = /opt/vmware/vspc log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.serial_port_proxy_uri = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.serial_port_service_uri = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.task_poll_interval = 0.5 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.use_linked_clone = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.vnc_keymap = en-us log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.vnc_port = 5900 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vmware.vnc_port_total = 10000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.auth_schemes = ['none'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.enabled = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.novncproxy_host = 0.0.0.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.novncproxy_port = 6080 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.server_listen = 127.0.0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.server_proxyclient_address = 127.0.0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.vencrypt_ca_certs = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.vencrypt_client_cert = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vnc.vencrypt_client_key = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.disable_fallback_pcpu_query = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.disable_group_policy_check_upcall = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.disable_libvirt_livesnapshot = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.disable_rootwrap = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.enable_numa_live_migration = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.handle_virt_lifecycle_events = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.libvirt_disable_apic = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.never_download_image_if_on_rbd = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] workarounds.reserve_disk_resource_for_image_cache = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.api_paste_config = api-paste.ini log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.client_socket_timeout = 900 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.default_pool_size = 1000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.keep_alive = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.max_header_line = 16384 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.secure_proxy_ssl_header = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.ssl_ca_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.475 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.ssl_cert_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.ssl_key_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.tcp_keepidle = 600 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] zvm.ca_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] zvm.cloud_connector_url = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] zvm.image_tmp_path = c:\openstack\build\nova\images log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] zvm.reachable_timeout = 300 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.enforce_new_defaults = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.enforce_scope = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.policy_default_rule = default log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.policy_dirs = ['policy.d'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.policy_file = c:\openstack\etc\policy.json log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.remote_content_type = application/x-www-form-urlencoded log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.remote_ssl_ca_crt_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.remote_ssl_client_crt_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.remote_ssl_client_key_file = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_policy.remote_ssl_verify_server_crt = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.connection_string = messaging:// log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.enabled = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.es_doc_type = notification log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.es_scroll_size = 10000 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.498 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.es_scroll_time = 2m log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.filter_error_trace = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.hmac_keys = SECRET_KEY log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.sentinel_service_name = mymaster log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.socket_timeout = 0.1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] profiler.trace_sqlalchemy = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] remote_debug.host = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] remote_debug.port = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.direct_mandatory_flag = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.enable_cancel_on_failover = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.heartbeat_in_pthread = True log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.514 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.ssl = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.ssl_ca_file = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.ssl_cert_file = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.ssl_key_file = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_rabbit.ssl_version = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_notifications.driver = [] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_reports.file_event_handler = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] oslo_reports.log_dir = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.capabilities = [12] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.group = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.helper_command = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.thread_pool_size = 6 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_linux_bridge_privileged.user = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.530 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.capabilities = [12] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.group = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.helper_command = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.thread_pool_size = 6 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] vif_plug_ovs_privileged.user = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.flat_interface = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.forward_bridge_interface = ['all'] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.iptables_bottom_regex = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.iptables_drop_action = DROP log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.iptables_top_regex = log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.network_device_mtu = 1500 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.use_ipv6 = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_linux_bridge.vlan_interface = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.isolate_vif = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.network_device_mtu = 1500 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.ovs_vsctl_timeout = 120 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.ovsdb_connection = tcp:127.0.0.1:6640 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.ovsdb_interface = native log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_vif_ovs.per_port_bridge = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.capabilities = [21] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.group = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.helper_command = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.logger_name = oslo_privsep.daemon log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.545 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.thread_pool_size = 6 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] privsep_osbrick.user = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.group = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.helper_command = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.logger_name = oslo_privsep.daemon log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.thread_pool_size = 6 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] nova_sys_admin.user = None log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_win.cache_temporary_wmi_objects = False log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_win.connect_cluster_timeout = 0 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_win.file_in_use_timeout = 15 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_win.hbaapi_lib_path = hbaapi.dll log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] os_win.wmi_job_terminate_timeout = 120 log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2611 2021-09-10 16:46:23.561 4076 3050945804776 MainThread DEBUG oslo_service.service [req-7fca6db6-afb0-4771-9072-83f51e95c8e2 - - - - -] ******************************************************************************** log_opt_values c:\python37\lib\site-packages\oslo_config\cfg.py:2613 2021-09-10 16:46:23.561 4076 3051051169400 GreenThread-1 INFO nova.service [-] Starting compute node (version 23.1.0) 2021-09-10 16:46:23.811 4076 3051051169400 GreenThread-1 WARNING nova.compute.manager [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Compute node n-h1-808474-1 not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host n-h1-808474-1 could not be found. 2021-09-10 16:46:23.921 4076 3051051169400 GreenThread-1 INFO nova.compute.manager [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host 2021-09-10 16:46:23.959 4076 3051051169400 GreenThread-1 WARNING nova.compute.manager [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] No compute node record found for host n-h1-808474-1. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host n-h1-808474-1 could not be found. 2021-09-10 16:46:23.959 4076 3051051169400 GreenThread-1 DEBUG oslo_concurrency.lockutils [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:46:23.959 4076 3051051169400 GreenThread-1 DEBUG oslo_concurrency.lockutils [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:46:23.959 4076 3051051169400 GreenThread-1 DEBUG nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:46:23.959 4076 3051051169400 GreenThread-1 DEBUG nova.virt.hyperv.hostops [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:46:23.975 4076 3051051169400 GreenThread-1 DEBUG nova.virt.hyperv.hostops [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:46:25.302 4076 3051051169400 GreenThread-1 DEBUG nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=14978MB free_disk=65GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:46:25.302 4076 3051051169400 GreenThread-1 DEBUG oslo_concurrency.lockutils [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:46:25.327 4076 3051051169400 GreenThread-1 WARNING nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] No compute node record for n-h1-808474-1:n-h1-808474-1: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host n-h1-808474-1 could not be found. 2021-09-10 16:46:25.343 4076 3051051169400 GreenThread-1 INFO nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Compute node record created for n-h1-808474-1:n-h1-808474-1 with uuid: 6729133d-6e37-4062-bd58-46fc02d92e9d 2021-09-10 16:46:25.444 4076 3051051169400 GreenThread-1 DEBUG nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Total usable vcpus: 6, total allocated vcpus: 0 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:46:25.444 4076 3051051169400 GreenThread-1 DEBUG nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=512MB phys_disk=79GB used_disk=0GB total_vcpus=6 used_vcpus=0 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:46:25.616 4076 3051051169400 GreenThread-1 INFO nova.scheduler.client.report [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] [req-e0c4934c-ad37-46de-a791-c87ea73bdafb] Created resource provider record via placement API for resource provider with UUID 6729133d-6e37-4062-bd58-46fc02d92e9d and name n-h1-808474-1. 2021-09-10 16:46:25.616 4076 3051051169400 GreenThread-1 DEBUG nova.virt.hyperv.hostops [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:46:25.655 4076 3051051169400 GreenThread-1 DEBUG nova.virt.hyperv.hostops [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:46:25.822 4076 3051051169400 GreenThread-1 DEBUG nova.compute.provider_tree [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0, 'reserved': 0}, 'MEMORY_MB': {'total': 16383, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5, 'reserved': 512}, 'DISK_GB': {'total': 79, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 16:46:25.890 4076 3051051169400 GreenThread-1 DEBUG nova.scheduler.client.report [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Updated inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with generation 0 in Placement from set_inventory_for_provider using data: {'VCPU': {'total': 6, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0, 'reserved': 0}, 'MEMORY_MB': {'total': 16383, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5, 'reserved': 512}, 'DISK_GB': {'total': 79, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:912 2021-09-10 16:46:25.890 4076 3051051169400 GreenThread-1 DEBUG nova.compute.provider_tree [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Updating resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d generation from 0 to 1 during operation: update_inventory _update_generation c:\openstack\build\nova\nova\compute\provider_tree.py:165 2021-09-10 16:46:25.890 4076 3051051169400 GreenThread-1 DEBUG nova.compute.provider_tree [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 16:46:25.969 4076 3051051169400 GreenThread-1 DEBUG nova.compute.provider_tree [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Updating resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d generation from 1 to 2 during operation: update_traits _update_generation c:\openstack\build\nova\nova\compute\provider_tree.py:165 2021-09-10 16:46:25.969 4076 3051051169400 GreenThread-1 DEBUG nova.compute.resource_tracker [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:46:25.969 4076 3051051169400 GreenThread-1 DEBUG oslo_concurrency.lockutils [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.672s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:46:25.969 4076 3051051169400 GreenThread-1 DEBUG nova.service [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Creating RPC server for service compute start c:\openstack\build\nova\nova\service.py:182 2021-09-10 16:46:26.000 4076 3051051169400 GreenThread-1 DEBUG nova.service [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] Join ServiceGroup membership for this service compute start c:\openstack\build\nova\nova\service.py:200 2021-09-10 16:46:26.000 4076 3051051169400 GreenThread-1 DEBUG nova.servicegroup.drivers.db [req-d527eeff-2082-454b-890b-1df9f762dddb - - - - -] DB_Driver: join new ServiceGroup member n-h1-808474-1 to the compute group, service = join c:\openstack\build\nova\nova\servicegroup\drivers\db.py:47 2021-09-10 16:47:16.000 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:16.016 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.820 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:47:21.820 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 16:47:21.834 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Didn't find any instances for network info cache update. _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9424 2021-09-10 16:47:21.834 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.834 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.834 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.851 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.891 4076 3051049642296 GreenThread-2 INFO nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running instance usage audit for host n-h1-808474-1 from 2021-09-10 15:00:00 to 2021-09-10 16:00:00. 0 instances. 2021-09-10 16:47:21.925 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.925 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.925 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:47:21.925 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:47:21.931 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:47:21.931 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:47:21.931 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:47:21.931 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:47:21.959 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:47:22.130 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=14904MB free_disk=65GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:47:22.130 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:47:22.242 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 0 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:47:22.242 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=512MB phys_disk=79GB used_disk=0GB total_vcpus=6 used_vcpus=0 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:47:22.278 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:47:22.292 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:47:22.469 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:47:22.490 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:47:22.490 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:47:22.490 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:48:22.455 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.455 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.487 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.487 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:48:22.487 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 16:48:22.515 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Didn't find any instances for network info cache update. _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9424 2021-09-10 16:48:22.515 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.525 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.799 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:48:22.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:22.826 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:48:22.826 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:48:22.826 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:48:22.826 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:48:22.838 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:48:23.031 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=14950MB free_disk=65GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:48:23.031 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:48:23.145 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 0 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:48:23.145 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=512MB phys_disk=79GB used_disk=0GB total_vcpus=6 used_vcpus=0 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:48:23.182 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:48:23.192 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:48:23.361 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:48:23.386 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:48:23.386 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:48:23.386 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:48:24.377 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:48:51.739 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:48:51.769 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:48:52.004 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:48:52.004 4076 3051049643000 MainThread DEBUG nova.virt.hardware [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:48:52.004 4076 3051049643000 MainThread INFO nova.compute.claims [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Claim successful on node n-h1-808474-1 2021-09-10 16:48:52.177 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.hostops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:48:52.193 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.hostops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:48:52.426 4076 3051049643000 MainThread DEBUG nova.compute.provider_tree [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:48:52.426 4076 3051049643000 MainThread DEBUG nova.scheduler.client.report [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:48:52.426 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.421s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:48:52.426 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:48:52.863 4076 3051049643000 MainThread DEBUG nova.compute.utils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:48:52.863 4076 3051049642472 GreenThread-3 DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:48:52.863 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:48:52.895 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:48:53.166 4076 3051049642472 GreenThread-3 WARNING oslo_policy.policy [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. 2021-09-10 16:48:53.166 4076 3051049642472 GreenThread-3 WARNING oslo_policy.policy [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. 2021-09-10 16:48:53.166 4076 3051049642472 GreenThread-3 WARNING oslo_policy.policy [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Policy Rules ['context_is_admin', 'admin_or_owner', 'admin_api'] specified in policy files are the same as the defaults provided by the service. You can remove these rules from policy files which will make maintenance easier. You can detect these redundant rules by ``oslopolicy-list-redundant`` tool also. 2021-09-10 16:48:53.207 4076 3051049642472 GreenThread-3 DEBUG nova.policy [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c07661d324e44af58805ab623c5b5ba7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8a87fac0b0b47888853a40ab4a2b355', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:48:53.222 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:48:53.239 4076 3051049643000 MainThread INFO nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Spawning new instance 2021-09-10 16:48:53.254 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "instance-00000002" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:48:53.254 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "instance-00000002" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:48:53.254 4076 3051049643000 MainThread DEBUG nova.block_device [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:48:53.254 4076 3051049643000 MainThread DEBUG os_win.utils.pathutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Creating directory: C:\openstack\instances\_base check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:48:53.254 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:02.175 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Successfully created port: c0c8e7e4-c81a-47f5-8d82-27a129730b8e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:49:06.394 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Successfully updated port: c0c8e7e4-c81a-47f5-8d82-27a129730b8e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:49:06.426 4076 3051049642472 GreenThread-3 DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:06.426 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:07.115 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:49:08.253 4076 3051054694472 MainThread DEBUG nova.compute.manager [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Received event network-changed-c0c8e7e4-c81a-47f5-8d82-27a129730b8e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:08.253 4076 3051054694472 MainThread DEBUG nova.compute.manager [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Refreshing instance network info cache due to event network-changed-c0c8e7e4-c81a-47f5-8d82-27a129730b8e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:08.724 4076 3051049642472 GreenThread-3 DEBUG nova.network.neutron [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:08.738 4076 3051049642472 GreenThread-3 DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:08.738 4076 3051049642472 GreenThread-3 DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Instance network_info: |[{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:49:08.738 4076 3051054694472 MainThread DEBUG oslo_concurrency.lockutils [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:08.738 4076 3051054694472 MainThread DEBUG nova.network.neutron [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Refreshing network info cache for port c0c8e7e4-c81a-47f5-8d82-27a129730b8e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:11.050 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 17.796s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:11.319 4076 3051049643000 MainThread DEBUG os_win.utils.pathutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Creating directory: C:\openstack\instances\instance-00000002 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:49:11.319 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000002\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:49:11.426 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Resizing VHD C:\openstack\instances\instance-00000002\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:49:11.458 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Preparing to wait for external event network-vif-plugged-c0c8e7e4-c81a-47f5-8d82-27a129730b8e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:49:11.458 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:11.458 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:11.458 4076 3051049643000 MainThread DEBUG os_win.utils.compute.vmutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Creating VM instance-00000002 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:49:11.574 4076 3051054694472 MainThread DEBUG nova.network.neutron [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated VIF entry in instance network info cache for port c0c8e7e4-c81a-47f5-8d82-27a129730b8e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:11.574 4076 3051054694472 MainThread DEBUG nova.network.neutron [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:11.584 4076 3051054694472 MainThread DEBUG oslo_concurrency.lockutils [req-d290ba73-4bb6-448d-b46c-f850a0720486 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:12.365 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:49:16.019 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:21.706 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:21.725 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:49:21.833 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:21.833 4076 3051049643880 MainThread DEBUG nova.virt.hardware [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:49:21.833 4076 3051049643880 MainThread INFO nova.compute.claims [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Claim successful on node n-h1-808474-1 2021-09-10 16:49:21.894 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:21.894 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:49:21.894 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 16:49:21.909 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Skipping network cache update for instance because it is Building. _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9352 2021-09-10 16:49:21.909 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Skipping network cache update for instance because it is Building. _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9352 2021-09-10 16:49:21.909 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Didn't find any instances for network info cache update. _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9424 2021-09-10 16:49:21.940 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.hostops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:21.958 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.hostops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:22.159 4076 3051049643880 MainThread DEBUG nova.compute.provider_tree [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:22.175 4076 3051049643880 MainThread DEBUG nova.scheduler.client.report [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:22.175 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:22.175 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:49:22.301 4076 3051049643880 MainThread DEBUG nova.compute.utils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:49:22.301 4076 3051049642472 GreenThread-4 DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:49:22.301 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:49:22.347 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:49:22.599 4076 3051049642472 GreenThread-4 DEBUG nova.policy [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '832a42f6b78c40c0bee881dab45d7c65', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa9a142ec08846a38fa7f3cbcd2b53d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:49:22.616 4076 3051049643880 MainThread INFO nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Spawning new instance 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG nova.block_device [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:22.616 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:22.644 4076 3051049643880 MainThread DEBUG os_win.utils.pathutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating directory: C:\openstack\instances\instance-00000005 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:49:22.644 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000005\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:49:22.678 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Resizing VHD C:\openstack\instances\instance-00000005\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:49:22.772 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-3e4ee351-c901-42cf-a5e7-a31537da6c33 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Received event network-vif-plugged-c0c8e7e4-c81a-47f5-8d82-27a129730b8e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:22.772 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-3e4ee351-c901-42cf-a5e7-a31537da6c33 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:22.772 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-3e4ee351-c901-42cf-a5e7-a31537da6c33 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:22.772 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-3e4ee351-c901-42cf-a5e7-a31537da6c33 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Processing event network-vif-plugged-c0c8e7e4-c81a-47f5-8d82-27a129730b8e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:49:22.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:22.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:22.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:22.803 4076 3051049643000 MainThread INFO nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Using config drive for instance 2021-09-10 16:49:22.831 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:22.831 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:22.831 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:49:22.831 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:22.853 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:23.058 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=14816MB free_disk=64GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:49:23.058 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:23.118 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:23.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:49:23.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:49:23.159 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:49:23.233 4076 3051049643000 MainThread INFO nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Creating config drive at C:\openstack\instances\instance-00000002\configdrive.iso 2021-09-10 16:49:23.272 4076 3051049643000 MainThread DEBUG oslo_concurrency.processutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000002\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprzf4w4nr execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:49:23.519 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 16:49:23.519 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 2 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:49:23.519 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=2560MB phys_disk=79GB used_disk=2GB total_vcpus=6 used_vcpus=2 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:49:23.565 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:23.594 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:23.741 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:23.754 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:49:23.803 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:23.819 4076 3051049643000 MainThread DEBUG oslo_concurrency.processutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000002\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprzf4w4nr" returned: 0 in 0.547s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:49:23.833 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:23.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:49:23.833 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.782s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:23.849 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.281s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:23.849 4076 3051055214664 MainThread DEBUG nova.virt.hardware [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:49:23.849 4076 3051055214664 MainThread INFO nova.compute.claims [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Claim successful on node n-h1-808474-1 2021-09-10 16:49:23.909 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Successfully created port: 298cc953-6875-4186-9b30-11fa49791716 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:49:24.003 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:24.038 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:24.066 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Setting boot order for instance: instance-00000002: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:49:24.132 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:49:24.179 4076 3051049643000 MainThread DEBUG os_win.utils.jobutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Waiting for WMI job: {'InstanceID': 'F262A519-618E-4337-B2EA-9A9D7D422005', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=28800), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:49:24.253 4076 3051055214664 MainThread DEBUG nova.compute.provider_tree [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:24.271 4076 3051055214664 MainThread DEBUG nova.scheduler.client.report [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:24.271 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:24.271 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:49:24.271 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:24.271 4076 3051055215544 MainThread DEBUG nova.virt.hardware [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:49:24.271 4076 3051055215544 MainThread INFO nova.compute.claims [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Claim successful on node n-h1-808474-1 2021-09-10 16:49:24.426 4076 3051055214664 MainThread DEBUG nova.compute.utils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:49:24.426 4076 3051049643704 GreenThread-5 DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:49:24.426 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:49:24.472 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:49:24.505 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.hostops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:24.505 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.hostops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:24.683 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:49:24.692 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Spawning new instance 2021-09-10 16:49:24.692 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:24.692 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:24.692 4076 3051055214664 MainThread DEBUG nova.block_device [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:49:24.692 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:24.692 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:24.714 4076 3051055214664 MainThread DEBUG os_win.utils.pathutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Creating directory: C:\openstack\instances\instance-00000006 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:49:24.728 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000006\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:49:24.728 4076 3051055215544 MainThread DEBUG nova.compute.provider_tree [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:24.739 4076 3051055215544 MainThread DEBUG nova.scheduler.client.report [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:24.739 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:24.739 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:49:24.759 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Resizing VHD C:\openstack\instances\instance-00000006\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:49:24.848 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:24.848 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:24.848 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:24.866 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:24.866 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:49:24.883 4076 3051055215544 MainThread DEBUG nova.compute.utils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:49:24.883 4076 3051055215192 GreenThread-6 DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:49:24.883 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:49:24.925 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:49:25.032 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Successfully updated port: 298cc953-6875-4186-9b30-11fa49791716 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:49:25.051 4076 3051049643704 GreenThread-5 DEBUG nova.policy [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66161d18b31e45309aee94f739bc6a3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145f8177efc74f5e9be31c5f74e69d0a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:49:25.072 4076 3051049642472 GreenThread-4 DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:25.072 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:25.098 4076 3051055215368 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:25.098 4076 3051055215368 MainThread INFO nova.compute.manager [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] VM Started (Lifecycle Event) 2021-09-10 16:49:25.098 4076 3051055215896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000002" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:25.114 4076 3051049643000 MainThread DEBUG os_win.utils.jobutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] WMI job succeeded: {'InstanceID': 'F262A519-618E-4337-B2EA-9A9D7D422005', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=895866), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:49:25.114 4076 3051049643000 MainThread DEBUG os_win.utils.compute.vmutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Successfully changed vm state of instance-00000002 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:49:25.114 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Successfully changed state of VM instance-00000002 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:49:25.114 4076 3051049643000 MainThread INFO nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Took 31.89 seconds to spawn the instance on the hypervisor. 2021-09-10 16:49:25.114 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:25.114 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:25.114 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:49:25.114 4076 3051055215896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000002" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:25.144 4076 3051055215544 MainThread INFO nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Spawning new instance 2021-09-10 16:49:25.144 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000007" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:25.144 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000007" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:25.144 4076 3051055215544 MainThread DEBUG nova.block_device [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:49:25.144 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:25.144 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:25.165 4076 3051055215544 MainThread DEBUG os_win.utils.pathutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating directory: C:\openstack\instances\instance-00000007 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:49:25.165 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000007\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:49:25.175 4076 3051055215192 GreenThread-6 DEBUG nova.policy [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '832a42f6b78c40c0bee881dab45d7c65', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa9a142ec08846a38fa7f3cbcd2b53d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:49:25.197 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-40243377-c0ec-468f-82ed-891a80dec0c2 - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:25.197 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-40243377-c0ec-468f-82ed-891a80dec0c2 - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:25.208 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Resizing VHD C:\openstack\instances\instance-00000007\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:49:25.239 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-40243377-c0ec-468f-82ed-891a80dec0c2 - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:25.287 4076 3051055215896 MainThread DEBUG nova.compute.manager [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-changed-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:25.287 4076 3051055215896 MainThread DEBUG nova.compute.manager [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Refreshing instance network info cache due to event network-changed-298cc953-6875-4186-9b30-11fa49791716. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:25.306 4076 3051055215368 MainThread INFO nova.compute.manager [req-40243377-c0ec-468f-82ed-891a80dec0c2 - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:49:25.334 4076 3051049643000 MainThread INFO nova.compute.manager [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Took 33.41 seconds to build instance. 2021-09-10 16:49:25.334 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:49:25.371 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-23f274a2-7b2f-4c0a-afbb-70022edb429f c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:25.820 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:49:25.897 4076 3051049642472 GreenThread-4 DEBUG nova.network.neutron [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:25.927 4076 3051049642472 GreenThread-4 DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:25.927 4076 3051049642472 GreenThread-4 DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Instance network_info: |[{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:49:25.927 4076 3051055215896 MainThread DEBUG oslo_concurrency.lockutils [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:25.927 4076 3051055215896 MainThread DEBUG nova.network.neutron [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Refreshing network info cache for port 298cc953-6875-4186-9b30-11fa49791716 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:25.927 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Preparing to wait for external event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:49:25.927 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:25.927 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:25.927 4076 3051049643880 MainThread DEBUG os_win.utils.compute.vmutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating VM instance-00000005 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:49:26.537 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:49:26.769 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Successfully created port: 54260358-c3ac-4d72-8f2e-1153615d8bdb _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:49:27.067 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Successfully created port: df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:49:27.117 4076 3051055215896 MainThread DEBUG nova.network.neutron [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated VIF entry in instance network info cache for port 298cc953-6875-4186-9b30-11fa49791716. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:27.117 4076 3051055215896 MainThread DEBUG nova.network.neutron [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:27.137 4076 3051055215896 MainThread DEBUG oslo_concurrency.lockutils [req-8cc9b7f6-e1e0-4c9a-8af9-dd36e20ced8b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:27.913 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Successfully updated port: 54260358-c3ac-4d72-8f2e-1153615d8bdb _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:49:27.945 4076 3051055215192 GreenThread-6 DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:27.945 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:28.162 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Successfully updated port: df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:49:28.195 4076 3051049643704 GreenThread-5 DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:28.195 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:28.195 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:49:28.305 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:49:28.396 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Received event network-changed-54260358-c3ac-4d72-8f2e-1153615d8bdb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:28.396 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Refreshing instance network info cache due to event network-changed-54260358-c3ac-4d72-8f2e-1153615d8bdb. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:28.457 4076 3051055215016 MainThread DEBUG nova.compute.manager [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Received event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:28.457 4076 3051055215016 MainThread DEBUG nova.compute.manager [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing instance network info cache due to event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:28.878 4076 3051049643704 GreenThread-5 DEBUG nova.network.neutron [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:28.898 4076 3051049643704 GreenThread-5 DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:28.898 4076 3051049643704 GreenThread-5 DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Instance network_info: |[{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:49:28.898 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Preparing to wait for external event network-vif-plugged-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:49:28.898 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:28.898 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:28.898 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Creating VM instance-00000006 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:49:28.898 4076 3051055215016 MainThread DEBUG oslo_concurrency.lockutils [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:28.898 4076 3051055215016 MainThread DEBUG nova.network.neutron [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:28.962 4076 3051055215192 GreenThread-6 DEBUG nova.network.neutron [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:29.011 4076 3051055215192 GreenThread-6 DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:29.011 4076 3051055215192 GreenThread-6 DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Instance network_info: |[{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:49:29.011 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:29.011 4076 3051049643352 MainThread DEBUG nova.network.neutron [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Refreshing network info cache for port 54260358-c3ac-4d72-8f2e-1153615d8bdb _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:29.011 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Preparing to wait for external event network-vif-plugged-54260358-c3ac-4d72-8f2e-1153615d8bdb prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:49:29.011 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:29.021 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:29.021 4076 3051055215544 MainThread DEBUG os_win.utils.compute.vmutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating VM instance-00000007 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:49:29.519 4076 3051055215016 MainThread DEBUG nova.network.neutron [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated VIF entry in instance network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:29.519 4076 3051055215016 MainThread DEBUG nova.network.neutron [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:29.534 4076 3051055215016 MainThread DEBUG oslo_concurrency.lockutils [req-c5d78f2f-84ca-4641-a8d7-938929057ae3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:29.581 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:49:29.726 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:49:30.006 4076 3051049643352 MainThread DEBUG nova.network.neutron [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated VIF entry in instance network info cache for port 54260358-c3ac-4d72-8f2e-1153615d8bdb. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:30.006 4076 3051049643352 MainThread DEBUG nova.network.neutron [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:30.034 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-9b342ea0-6889-42a3-b7fd-f3ca88054371 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:33.994 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-74e859de-1bc3-4c57-a8e4-fc2357e955d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:33.994 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-74e859de-1bc3-4c57-a8e4-fc2357e955d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:33.994 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-74e859de-1bc3-4c57-a8e4-fc2357e955d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:34.016 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-74e859de-1bc3-4c57-a8e4-fc2357e955d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Processing event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:49:34.053 4076 3051049643880 MainThread INFO nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Using config drive for instance 2021-09-10 16:49:34.288 4076 3051049643880 MainThread INFO nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating config drive at C:\openstack\instances\instance-00000005\configdrive.iso 2021-09-10 16:49:34.331 4076 3051049643880 MainThread DEBUG oslo_concurrency.processutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpv2ci2_d8 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:49:34.654 4076 3051049643880 MainThread DEBUG oslo_concurrency.processutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpv2ci2_d8" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:49:34.901 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Setting boot order for instance: instance-00000005: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:49:34.936 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:49:34.976 4076 3051049643880 MainThread DEBUG os_win.utils.jobutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '570F5DAF-D0A1-4CA8-A678-95043B4FD6D0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19290), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:49:35.243 4076 3051049643000 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:35.243 4076 3051049643000 MainThread INFO nova.compute.manager [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] VM Started (Lifecycle Event) 2021-09-10 16:49:35.268 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:35.289 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-63484e1d-f94c-487a-a702-dd791237c3e9 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:35.289 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.vmops [req-63484e1d-f94c-487a-a702-dd791237c3e9 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:35.302 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:35.321 4076 3051049643880 MainThread DEBUG os_win.utils.jobutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '570F5DAF-D0A1-4CA8-A678-95043B4FD6D0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=273347), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:49:35.321 4076 3051049643880 MainThread DEBUG os_win.utils.compute.vmutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000005 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:49:35.321 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000005 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:49:35.321 4076 3051049643880 MainThread INFO nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Took 12.70 seconds to spawn the instance on the hypervisor. 2021-09-10 16:49:35.321 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:35.342 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:35.342 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-63484e1d-f94c-487a-a702-dd791237c3e9 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:35.379 4076 3051049643000 MainThread INFO nova.compute.manager [req-63484e1d-f94c-487a-a702-dd791237c3e9 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:49:35.522 4076 3051049643880 MainThread INFO nova.compute.manager [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Took 13.73 seconds to build instance. 2021-09-10 16:49:35.566 4076 3051049643880 MainThread DEBUG oslo_concurrency.lockutils [req-a637801d-7998-47eb-bbf6-2a25df13b035 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 13.875s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:35.615 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] check_can_live_migrate_destination called check_can_live_migrate_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:132 2021-09-10 16:49:35.615 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.pathutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Checking if C:\openstack\instances and \\n-h2-808474-1\C$\openstack\instances point to the same location. check_dirs_shared_storage c:\openstack\build\nova\nova\virt\hyperv\pathutils.py:183 2021-09-10 16:49:35.633 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] destination check data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=,migration=,old_vol_attachment_ids=,vifs=,wait_for_vif_plugged=) check_can_live_migrate_destination c:\openstack\build\nova\nova\compute\manager.py:8040 2021-09-10 16:49:35.662 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "compute-rpcapi-router" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:35.662 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "compute-rpcapi-router" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:35.753 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] cleanup_live_migration_destination_check called cleanup_live_migration_destination_check c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:148 2021-09-10 16:49:36.584 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids=,vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8168 2021-09-10 16:49:36.612 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:36.612 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Received event network-vif-plugged-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Processing event network-vif-plugged-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Received event network-vif-plugged-54260358-c3ac-4d72-8f2e-1153615d8bdb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:36.878 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-122f0b86-67df-496a-beb0-2230aa06bb83 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Processing event network-vif-plugged-54260358-c3ac-4d72-8f2e-1153615d8bdb _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:49:36.941 4076 3051055215544 MainThread INFO nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Using config drive for instance 2021-09-10 16:49:36.941 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Using config drive for instance 2021-09-10 16:49:37.137 4076 3051055215544 MainThread INFO nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Creating config drive at C:\openstack\instances\instance-00000007\configdrive.iso 2021-09-10 16:49:37.175 4076 3051055215544 MainThread DEBUG oslo_concurrency.processutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000007\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpnadfr0et execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:49:37.318 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Creating config drive at C:\openstack\instances\instance-00000006\configdrive.iso 2021-09-10 16:49:37.348 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000006\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp337pb1ma execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:49:37.537 4076 3051055215544 MainThread DEBUG oslo_concurrency.processutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000007\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpnadfr0et" returned: 0 in 0.360s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:49:37.787 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000006\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp337pb1ma" returned: 0 in 0.438s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:49:37.816 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Setting boot order for instance: instance-00000007: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:49:37.873 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:49:37.914 4076 3051055215544 MainThread DEBUG os_win.utils.jobutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': 'B88CCED5-8C0D-4599-9A94-098CFFF7E3C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18039), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:49:38.052 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Setting boot order for instance: instance-00000006: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:49:38.133 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:49:38.161 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '3820F69C-DF5A-462F-905C-5973F43EDE29', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=28955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:49:38.190 4076 3051049643880 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:38.190 4076 3051049643880 MainThread INFO nova.compute.manager [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] VM Started (Lifecycle Event) 2021-09-10 16:49:38.190 4076 3051055215192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000007" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:38.211 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-767eb32c-d133-4d6f-a9a3-290d35d18973 - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:38.211 4076 3051049643880 MainThread DEBUG nova.virt.hyperv.vmops [req-767eb32c-d133-4d6f-a9a3-290d35d18973 - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:38.230 4076 3051055215192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000007" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:38.257 4076 3051049643880 MainThread DEBUG nova.compute.manager [req-767eb32c-d133-4d6f-a9a3-290d35d18973 - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:38.286 4076 3051055215544 MainThread DEBUG os_win.utils.jobutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': 'B88CCED5-8C0D-4599-9A94-098CFFF7E3C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=264244), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:49:38.286 4076 3051055215544 MainThread DEBUG os_win.utils.compute.vmutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000007 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:49:38.286 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000007 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:49:38.286 4076 3051055215544 MainThread INFO nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Took 13.17 seconds to spawn the instance on the hypervisor. 2021-09-10 16:49:38.286 4076 3051055215544 MainThread DEBUG nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:38.286 4076 3051055215544 MainThread DEBUG nova.virt.hyperv.vmops [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:38.305 4076 3051049643880 MainThread INFO nova.compute.manager [req-767eb32c-d133-4d6f-a9a3-290d35d18973 - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:49:38.348 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:38.379 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:38.379 4076 3051049643000 MainThread DEBUG os_brick.utils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:49:38.494 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job succeeded: {'InstanceID': '3820F69C-DF5A-462F-905C-5973F43EDE29', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=285909), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:49:38.494 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Successfully changed vm state of instance-00000006 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:49:38.494 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Successfully changed state of VM instance-00000006 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:49:38.494 4076 3051055214664 MainThread INFO nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Took 13.81 seconds to spawn the instance on the hypervisor. 2021-09-10 16:49:38.494 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:38.494 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:38.494 4076 3051049643000 MainThread DEBUG os_brick.utils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] <== get_connector_properties: return (114ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:49:38.494 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] pre_live_migration called pre_live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:92 2021-09-10 16:49:38.516 4076 3051055215544 MainThread INFO nova.compute.manager [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Took 14.69 seconds to build instance. 2021-09-10 16:49:38.521 4076 3051055215016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:38.521 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:38.521 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] VM Started (Lifecycle Event) 2021-09-10 16:49:38.550 4076 3051049643000 MainThread DEBUG nova.block_device [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:49:38.550 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:38.550 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:38.550 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] driver pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={},vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8230 2021-09-10 16:49:38.568 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-7787cff5-3f64-4093-bd6c-6995f7fde0db - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:38.568 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-7787cff5-3f64-4093-bd6c-6995f7fde0db - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:38.568 4076 3051055215544 MainThread DEBUG oslo_concurrency.lockutils [req-74120097-2922-49b7-8b0e-7f7005237e9d 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 14.828s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:38.568 4076 3051055215016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:38.628 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-7787cff5-3f64-4093-bd6c-6995f7fde0db - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:38.674 4076 3051055214664 MainThread INFO nova.compute.manager [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Took 15.16 seconds to build instance. 2021-09-10 16:49:38.723 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-d950cd93-b3f5-4dc1-80a6-95f9d9b8ea70 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 15.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:40.600 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Port 5cc5f59c-ef6e-4ba0-b041-e283e7d8339a updated with migration profile {'migrating_to': 'n-h1-808474-1'} successfully _setup_migration_port_profile c:\openstack\build\nova\nova\network\neutron.py:343 2021-09-10 16:49:40.600 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration result data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={},vifs=[VIFMigrateData],wait_for_vif_plugged=True) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8285 2021-09-10 16:49:41.677 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:41.677 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:41.677 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:41.677 4076 3051049643000 MainThread ERROR nova.compute.manager [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Traceback (most recent call last): 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] self._delete_instance(context, instance, bdms) 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] NameError: name 'instant' is not defined 2021-09-10 16:49:41.677 4076 3051049643000 MainThread TRACE nova.compute.manager [instance: 7ed088c4-2147-476f-b179-befc33eb129d] 2021-09-10 16:49:41.925 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:42.193 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:42.256 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.hostops [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:42.256 4076 3051049643000 MainThread DEBUG nova.virt.hyperv.hostops [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:42.490 4076 3051049643000 MainThread DEBUG nova.compute.provider_tree [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:42.508 4076 3051049643000 MainThread DEBUG nova.scheduler.client.report [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:42.508 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:42.519 4076 3051049643000 MainThread INFO nova.compute.manager [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:49:42.519 4076 3051049643000 MainThread ERROR oslo_messaging.rpc.server [req-1f9c4f67-f3fb-401b-8d75-813accadd076 c07661d324e44af58805ab623c5b5ba7 f8a87fac0b0b47888853a40ab4a2b355 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:49:42.519 4076 3051049643000 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:49:42.585 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:42.628 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:49:42.785 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:42.785 4076 3051049643352 MainThread DEBUG nova.virt.hardware [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:49:42.785 4076 3051049643352 MainThread INFO nova.compute.claims [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Claim successful on node n-h1-808474-1 2021-09-10 16:49:42.941 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:49:42.963 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:49:43.081 4076 3051049643000 MainThread INFO nova.compute.manager [req-1dc2ad4f-2878-4afe-845e-c19833b2db88 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:43.081 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1dc2ad4f-2878-4afe-845e-c19833b2db88 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:43.081 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-1dc2ad4f-2878-4afe-845e-c19833b2db88 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:43.145 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Received event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:43.145 4076 3051049643000 MainThread DEBUG nova.compute.manager [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing instance network info cache due to event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:43.162 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:43.162 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:43.209 4076 3051049643352 MainThread DEBUG nova.compute.provider_tree [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:49:43.224 4076 3051049643352 MainThread DEBUG nova.scheduler.client.report [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:49:43.224 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:43.224 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:49:43.335 4076 3051049643352 MainThread DEBUG nova.compute.utils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:49:43.335 4076 3051055215016 GreenThread-7 DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:49:43.335 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:49:43.349 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:49:43.628 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:49:43.628 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Spawning new instance 2021-09-10 16:49:43.662 4076 3051055215016 GreenThread-7 DEBUG nova.policy [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '832a42f6b78c40c0bee881dab45d7c65', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa9a142ec08846a38fa7f3cbcd2b53d8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:49:43.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-0000000a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:43.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-0000000a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:43.662 4076 3051049643352 MainThread DEBUG nova.block_device [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:49:43.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:43.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:43.682 4076 3051049643352 MainThread DEBUG os_win.utils.pathutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating directory: C:\openstack\instances\instance-0000000a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:49:43.682 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000000a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:49:43.724 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Resizing VHD C:\openstack\instances\instance-0000000a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:49:44.681 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Successfully created port: 0519e0a6-143b-4dde-87d3-52c308238e7f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:49:45.365 4076 3051055217304 MainThread INFO nova.compute.manager [req-07ee0033-1b34-4af7-a8c4-e25a64f7a9d2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:45.365 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-07ee0033-1b34-4af7-a8c4-e25a64f7a9d2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:45.365 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-07ee0033-1b34-4af7-a8c4-e25a64f7a9d2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:45.879 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated VIF entry in instance network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:45.879 4076 3051049643000 MainThread DEBUG nova.network.neutron [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.25.0.88", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:45.926 4076 3051049643000 MainThread DEBUG oslo_concurrency.lockutils [req-de891cd6-53c7-45a8-b3d9-af5e5d7e1975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:45.949 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Successfully updated port: 0519e0a6-143b-4dde-87d3-52c308238e7f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:49:45.990 4076 3051055215016 GreenThread-7 DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:45.990 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:46.119 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:46.119 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] VM Started (Lifecycle Event) 2021-09-10 16:49:46.119 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000004" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:46.144 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-31eb6b98-dc64-436d-bc8e-7478791c3de3 - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:46.161 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-31eb6b98-dc64-436d-bc8e-7478791c3de3 - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:46.161 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:49:46.193 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000004" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:46.223 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-31eb6b98-dc64-436d-bc8e-7478791c3de3 - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: migrating, current DB power_state: 1, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:46.269 4076 3051055217304 MainThread INFO nova.compute.manager [req-31eb6b98-dc64-436d-bc8e-7478791c3de3 - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] During the sync_power process the instance has moved from host n-h2-808474-1 to host n-h1-808474-1 2021-09-10 16:49:46.441 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Received event network-changed-0519e0a6-143b-4dde-87d3-52c308238e7f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:46.441 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Refreshing instance network info cache due to event network-changed-0519e0a6-143b-4dde-87d3-52c308238e7f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:49:46.538 4076 3051055217128 MainThread INFO nova.compute.manager [req-f3e393de-cd13-4bea-9452-975572d38045 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:46.538 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-f3e393de-cd13-4bea-9452-975572d38045 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:46.550 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-f3e393de-cd13-4bea-9452-975572d38045 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:46.790 4076 3051055215016 GreenThread-7 DEBUG nova.network.neutron [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:46.821 4076 3051055215016 GreenThread-7 DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:46.821 4076 3051055215016 GreenThread-7 DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Instance network_info: |[{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:49:46.821 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:46.821 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Refreshing network info cache for port 0519e0a6-143b-4dde-87d3-52c308238e7f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:49:46.821 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Preparing to wait for external event network-vif-plugged-0519e0a6-143b-4dde-87d3-52c308238e7f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:49:46.837 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:46.837 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:46.837 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating VM instance-0000000a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:49:47.649 4076 3051055217128 MainThread INFO nova.compute.manager [req-dbf1963f-e8bd-4bb3-88b9-a6cc7da7a986 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:47.667 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-dbf1963f-e8bd-4bb3-88b9-a6cc7da7a986 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:47.667 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-dbf1963f-e8bd-4bb3-88b9-a6cc7da7a986 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:47.757 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:49:47.879 4076 3051055217128 MainThread INFO nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Post operation of migration started 2021-09-10 16:49:47.956 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated VIF entry in instance network info cache for port 0519e0a6-143b-4dde-87d3-52c308238e7f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:49:47.956 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:48.010 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-fb8dd3cd-dece-4a0b-82ae-200d708be647 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:48.068 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:49:48.068 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:49:49.057 4076 3051055217480 MainThread INFO nova.compute.manager [req-04391d24-9979-4e15-bf90-2fbdff8bbaaa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:49.057 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-04391d24-9979-4e15-bf90-2fbdff8bbaaa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:49.057 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-04391d24-9979-4e15-bf90-2fbdff8bbaaa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:49.225 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:49:49.253 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:49:49.278 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:49.278 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:49.278 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] post_live_migration_at_destination called post_live_migration_at_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:124 2021-09-10 16:49:49.278 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:49.278 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:49.369 4076 3051055217128 MainThread DEBUG nova.objects.instance [req-746bde86-693a-418e-8dd5-2f2943af58a4 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:49:50.521 4076 3051055217128 MainThread INFO nova.compute.manager [req-32c9a00b-82ab-433b-a61e-f01289f23320 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:50.521 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-32c9a00b-82ab-433b-a61e-f01289f23320 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:50.521 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-32c9a00b-82ab-433b-a61e-f01289f23320 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:52.319 4076 3051055217128 MainThread INFO nova.compute.manager [req-a8161e92-14cb-488a-bbe5-ea8a4045dcd5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:52.319 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-a8161e92-14cb-488a-bbe5-ea8a4045dcd5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:52.319 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-a8161e92-14cb-488a-bbe5-ea8a4045dcd5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:52.899 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-7ea16425-ee0b-4f9f-8b97-56d031110f75 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Received event network-vif-plugged-0519e0a6-143b-4dde-87d3-52c308238e7f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:49:52.899 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-7ea16425-ee0b-4f9f-8b97-56d031110f75 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:52.899 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-7ea16425-ee0b-4f9f-8b97-56d031110f75 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:52.899 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-7ea16425-ee0b-4f9f-8b97-56d031110f75 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Processing event network-vif-plugged-0519e0a6-143b-4dde-87d3-52c308238e7f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:49:52.944 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Using config drive for instance 2021-09-10 16:49:53.128 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Creating config drive at C:\openstack\instances\instance-0000000a\configdrive.iso 2021-09-10 16:49:53.195 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000000a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpha44p88j execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:49:53.503 4076 3051055214664 MainThread INFO nova.compute.manager [req-2f650f91-ee33-4575-9b73-406b09e5fe10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:53.503 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-2f650f91-ee33-4575-9b73-406b09e5fe10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:53.503 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-2f650f91-ee33-4575-9b73-406b09e5fe10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:53.596 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000000a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpha44p88j" returned: 0 in 0.406s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:49:53.957 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Setting boot order for instance: instance-0000000a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:49:54.035 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:49:54.081 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '43CDEB2B-3965-4F50-AB04-47E004687A40', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18245), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:49:54.456 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '43CDEB2B-3965-4F50-AB04-47E004687A40', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=376949), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:49:54.456 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-0000000a to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:49:54.456 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-0000000a to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:49:54.456 4076 3051049643352 MainThread INFO nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Took 10.83 seconds to spawn the instance on the hypervisor. 2021-09-10 16:49:54.456 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:54.456 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:54.476 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:49:54.476 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] VM Started (Lifecycle Event) 2021-09-10 16:49:54.476 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:54.505 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-36551598-9c7a-4389-8f79-366c43e7038c - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:49:54.505 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-36551598-9c7a-4389-8f79-366c43e7038c - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:49:54.527 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.063s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:54.597 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-36551598-9c7a-4389-8f79-366c43e7038c - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:49:54.645 4076 3051055217304 MainThread INFO nova.compute.manager [req-36551598-9c7a-4389-8f79-366c43e7038c - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:49:54.675 4076 3051049643352 MainThread INFO nova.compute.manager [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Took 11.94 seconds to build instance. 2021-09-10 16:49:54.707 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-b7208085-4d6f-47c6-aed5-982ca8ee13b1 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 12.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:54.721 4076 3051049643352 MainThread INFO nova.compute.manager [req-3acb1032-99ea-4ad5-8355-9ca46fcad910 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:54.721 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-3acb1032-99ea-4ad5-8355-9ca46fcad910 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:54.721 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-3acb1032-99ea-4ad5-8355-9ca46fcad910 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:55.899 4076 3051049643352 MainThread INFO nova.compute.manager [req-8514aa3f-13f1-486c-ba59-1f448d910174 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:55.899 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-8514aa3f-13f1-486c-ba59-1f448d910174 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:55.914 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-8514aa3f-13f1-486c-ba59-1f448d910174 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:57.065 4076 3051049643352 MainThread INFO nova.compute.manager [req-dc19380d-25c5-4c0f-bbc6-213e86d8e38e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:57.081 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-dc19380d-25c5-4c0f-bbc6-213e86d8e38e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:57.081 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-dc19380d-25c5-4c0f-bbc6-213e86d8e38e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:58.226 4076 3051049643352 MainThread INFO nova.compute.manager [req-9827a817-ae17-4eee-8285-dcc1ad22cab7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:58.237 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-9827a817-ae17-4eee-8285-dcc1ad22cab7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:58.237 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-9827a817-ae17-4eee-8285-dcc1ad22cab7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:49:59.346 4076 3051049643352 MainThread INFO nova.compute.manager [req-479a1191-b148-4866-956b-f544e6b572cc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:49:59.346 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-479a1191-b148-4866-956b-f544e6b572cc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:49:59.346 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-479a1191-b148-4866-956b-f544e6b572cc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:00.049 4076 3051049643352 MainThread INFO nova.compute.manager [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Rebuilding instance 2021-09-10 16:50:00.145 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'trusted_certs' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:00.177 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:00.177 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:00.335 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'pci_requests' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:00.347 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'pci_devices' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:00.365 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'resources' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:00.383 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'migration_context' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:00.394 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:50:00.394 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:50:00.394 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:00.411 4076 3051049643352 MainThread INFO nova.virt.hyperv.serialconsoleops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Stopping instance instance-00000005 serial console handler. 2021-09-10 16:50:00.411 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:00.411 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Performing Soft shutdown on instance _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:778 2021-09-10 16:50:00.411 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 60 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:00.440 4076 3051049642472 MainThread INFO nova.compute.manager [req-0978312c-640b-4599-80d5-93756c149999 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:00.440 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0978312c-640b-4599-80d5-93756c149999 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:00.440 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0978312c-640b-4599-80d5-93756c149999 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:01.599 4076 3051055217304 MainThread INFO nova.compute.manager [req-19506606-81f5-4932-b4cf-2c105c5be508 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:01.599 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-19506606-81f5-4932-b4cf-2c105c5be508 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:01.612 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-19506606-81f5-4932-b4cf-2c105c5be508 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:02.691 4076 3051055217304 MainThread INFO nova.compute.manager [req-fdcaa4f3-09c5-4995-8d55-88d8b943d16c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:02.691 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-fdcaa4f3-09c5-4995-8d55-88d8b943d16c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:02.691 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-fdcaa4f3-09c5-4995-8d55-88d8b943d16c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:03.801 4076 3051055217304 MainThread INFO nova.compute.manager [req-b60a861d-4696-41d6-834e-d9ba74ddb2af 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:03.801 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-b60a861d-4696-41d6-834e-d9ba74ddb2af 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:03.817 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-b60a861d-4696-41d6-834e-d9ba74ddb2af 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:04.896 4076 3051055217304 MainThread INFO nova.compute.manager [req-73a313cc-f64c-4a6e-b24f-6892705053f7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:04.896 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-73a313cc-f64c-4a6e-b24f-6892705053f7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:04.909 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-73a313cc-f64c-4a6e-b24f-6892705053f7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:06.258 4076 3051055217304 MainThread INFO nova.compute.manager [req-67b22b58-6432-40b6-be8f-7991571b91b3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:06.258 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-67b22b58-6432-40b6-be8f-7991571b91b3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:06.258 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-67b22b58-6432-40b6-be8f-7991571b91b3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:07.393 4076 3051055217304 MainThread INFO nova.compute.manager [req-a858ff9d-c8d0-487c-ba7c-0dd4198a07a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:07.410 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-a858ff9d-c8d0-487c-ba7c-0dd4198a07a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:07.410 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-a858ff9d-c8d0-487c-ba7c-0dd4198a07a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:07.955 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:07.955 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:07.955 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:50:07.955 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] VM Stopped (Lifecycle Event) 2021-09-10 16:50:08.003 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e5fa54c4-182c-4ced-8e58-bc4f1419b4b6 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:08.003 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e5fa54c4-182c-4ced-8e58-bc4f1419b4b6 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:08.065 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e5fa54c4-182c-4ced-8e58-bc4f1419b4b6 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: error, current task_state: rebuilding, current DB power_state: 1, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:50:08.102 4076 3051055217304 MainThread INFO nova.compute.manager [req-e5fa54c4-182c-4ced-8e58-bc4f1419b4b6 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] During sync_power_state the instance has a pending task (rebuilding). Skip. 2021-09-10 16:50:09.026 4076 3051055214664 MainThread INFO nova.compute.manager [req-0b415bf5-acb9-4b92-acca-00ecca96386f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:09.026 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-0b415bf5-acb9-4b92-acca-00ecca96386f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:09.026 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-0b415bf5-acb9-4b92-acca-00ecca96386f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:09.034 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] check_can_live_migrate_destination called check_can_live_migrate_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:132 2021-09-10 16:50:09.034 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.pathutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Checking if C:\openstack\instances and \\n-h2-808474-1\C$\openstack\instances point to the same location. check_dirs_shared_storage c:\openstack\build\nova\nova\virt\hyperv\pathutils.py:183 2021-09-10 16:50:09.050 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] destination check data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=,migration=,old_vol_attachment_ids=,vifs=,wait_for_vif_plugged=) check_can_live_migrate_destination c:\openstack\build\nova\nova\compute\manager.py:8040 2021-09-10 16:50:09.253 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] cleanup_live_migration_destination_check called cleanup_live_migration_destination_check c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:148 2021-09-10 16:50:10.253 4076 3051055217304 MainThread INFO nova.compute.manager [req-41d9c59f-16f1-45b1-a9a0-bbd1c3f2b2dc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:10.253 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-41d9c59f-16f1-45b1-a9a0-bbd1c3f2b2dc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:10.271 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-41d9c59f-16f1-45b1-a9a0-bbd1c3f2b2dc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:10.457 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 50 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:10.565 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown succeeded. 2021-09-10 16:50:10.565 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Got request to destroy instance 2021-09-10 16:50:11.348 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids=,vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8168 2021-09-10 16:50:11.396 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:11.396 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:50:11.677 4076 3051055217128 MainThread INFO nova.compute.manager [req-17f968d8-fe77-4c91-8103-2e4793a56991 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:11.677 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-17f968d8-fe77-4c91-8103-2e4793a56991 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:11.677 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-17f968d8-fe77-4c91-8103-2e4793a56991 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:11.990 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:50:11.990 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:11.990 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.019 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000005 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:50:12.019 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000005 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:50:12.080 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '06826C16-F3FA-4551-9D46-4E5BEC0C40F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17862), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:50:12.208 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '06826C16-F3FA-4551-9D46-4E5BEC0C40F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=60568), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:50:12.208 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:12.208 4076 3051049643352 MainThread DEBUG os_win.utils.pathutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Removing directory: C:\openstack\instances\instance-00000005 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:50:12.208 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.456 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:12.492 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:12.492 4076 3051049642472 MainThread DEBUG os_brick.utils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:50:12.505 4076 3051049642472 MainThread DEBUG os_brick.utils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] <== get_connector_properties: return (13ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:50:12.505 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] pre_live_migration called pre_live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:92 2021-09-10 16:50:12.521 4076 3051049642472 MainThread DEBUG nova.block_device [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:12.533 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:12.533 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.533 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] driver pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={},vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8230 2021-09-10 16:50:12.533 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Spawning new instance 2021-09-10 16:50:12.551 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:12.551 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.551 4076 3051049643352 MainThread DEBUG nova.block_device [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:12.551 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:12.551 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.580 4076 3051049643352 MainThread DEBUG os_win.utils.pathutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating directory: C:\openstack\instances\instance-00000005 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:50:12.580 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000005\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:50:12.643 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Resizing VHD C:\openstack\instances\instance-00000005\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:50:12.693 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Preparing to wait for external event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:50:12.693 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:12.693 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:12.693 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating VM instance-00000005 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:50:13.018 4076 3051055214664 MainThread INFO nova.compute.manager [req-bfc278e7-c576-4dc7-babc-75fd7436f7a9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:13.018 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-bfc278e7-c576-4dc7-babc-75fd7436f7a9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:13.018 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-bfc278e7-c576-4dc7-babc-75fd7436f7a9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:13.537 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Port 02ca13c3-f2a9-4341-b400-dd8762c5f142 updated with migration profile {'migrating_to': 'n-h1-808474-1'} successfully _setup_migration_port_profile c:\openstack\build\nova\nova\network\neutron.py:343 2021-09-10 16:50:13.537 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration result data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={},vifs=[VIFMigrateData],wait_for_vif_plugged=True) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8285 2021-09-10 16:50:13.582 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:50:14.209 4076 3051049642472 MainThread INFO nova.compute.manager [req-b687f958-4b52-4fbb-bda4-8d6ce8010572 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:14.209 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-b687f958-4b52-4fbb-bda4-8d6ce8010572 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:14.209 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-b687f958-4b52-4fbb-bda4-8d6ce8010572 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:15.428 4076 3051049642472 MainThread INFO nova.compute.manager [req-425eda74-bcae-40cd-becb-7393b56a49b0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:15.445 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-425eda74-bcae-40cd-becb-7393b56a49b0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:15.445 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-425eda74-bcae-40cd-becb-7393b56a49b0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:16.612 4076 3051049642472 MainThread INFO nova.compute.manager [req-c7209957-a4b4-411f-81c2-d7f4c3501d19 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:16.612 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c7209957-a4b4-411f-81c2-d7f4c3501d19 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:16.612 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c7209957-a4b4-411f-81c2-d7f4c3501d19 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:17.332 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-faf4d58d-6fd9-4e0e-a7b9-b3c5eec49872 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:17.351 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-faf4d58d-6fd9-4e0e-a7b9-b3c5eec49872 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:17.351 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-faf4d58d-6fd9-4e0e-a7b9-b3c5eec49872 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:17.351 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-faf4d58d-6fd9-4e0e-a7b9-b3c5eec49872 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] No event matching network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 in dict_keys([('network-vif-plugged', '298cc953-6875-4186-9b30-11fa49791716')]) pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:330 2021-09-10 16:50:17.351 4076 3051049642472 MainThread WARNING nova.compute.manager [req-faf4d58d-6fd9-4e0e-a7b9-b3c5eec49872 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received unexpected event network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 for instance with vm_state error and task_state rebuild_spawning. 2021-09-10 16:50:17.909 4076 3051049642472 MainThread INFO nova.compute.manager [req-c8493030-aed9-4848-8697-48ce35e065eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:17.925 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c8493030-aed9-4848-8697-48ce35e065eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:17.925 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c8493030-aed9-4848-8697-48ce35e065eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:19.022 4076 3051049642472 MainThread INFO nova.compute.manager [req-062504cf-7769-43b8-a68e-5157a9250183 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:19.022 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-062504cf-7769-43b8-a68e-5157a9250183 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:19.022 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-062504cf-7769-43b8-a68e-5157a9250183 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:19.787 4076 3051049642472 MainThread INFO nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Post operation of migration started 2021-09-10 16:50:19.913 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:19.913 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:50:20.111 4076 3051055216952 MainThread INFO nova.compute.manager [req-a3fb2a99-bddf-456b-a5d9-0933d08a0145 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:20.111 4076 3051055216952 MainThread DEBUG oslo_concurrency.lockutils [req-a3fb2a99-bddf-456b-a5d9-0933d08a0145 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:20.111 4076 3051055216952 MainThread DEBUG oslo_concurrency.lockutils [req-a3fb2a99-bddf-456b-a5d9-0933d08a0145 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:20.596 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:20.620 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:20.646 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:20.646 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:20.646 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] post_live_migration_at_destination called post_live_migration_at_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:124 2021-09-10 16:50:20.646 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:20.646 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.vmops [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:20.724 4076 3051049642472 MainThread DEBUG nova.objects.instance [req-8b838353-e342-487e-8147-64d0ab7cf83e 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:50:21.146 4076 3051055217656 MainThread DEBUG nova.compute.manager [req-a175a1f8-a2a5-4b6e-913c-59485c73c53f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:21.146 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-a175a1f8-a2a5-4b6e-913c-59485c73c53f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:21.146 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-a175a1f8-a2a5-4b6e-913c-59485c73c53f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:21.146 4076 3051055217656 MainThread DEBUG nova.compute.manager [req-a175a1f8-a2a5-4b6e-913c-59485c73c53f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Processing event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:50:21.193 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Using config drive for instance 2021-09-10 16:50:21.205 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'ec2_ids' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:21.224 4076 3051055217656 MainThread INFO nova.compute.manager [req-257bd24b-c4bb-4c14-9f5f-26fcec2cf04b 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:21.224 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-257bd24b-c4bb-4c14-9f5f-26fcec2cf04b 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:21.224 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-257bd24b-c4bb-4c14-9f5f-26fcec2cf04b 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:21.288 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'keypairs' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:21.349 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'device_metadata' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:21.865 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating config drive at C:\openstack\instances\instance-00000005\configdrive.iso 2021-09-10 16:50:21.908 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy15n2ac9 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:50:22.237 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy15n2ac9" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:50:22.365 4076 3051049643704 MainThread INFO nova.compute.manager [req-2a117a8b-4af2-4995-9cbc-26ab2b321ccf 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:22.365 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-2a117a8b-4af2-4995-9cbc-26ab2b321ccf 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:22.365 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-2a117a8b-4af2-4995-9cbc-26ab2b321ccf 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:22.597 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Setting boot order for instance: instance-00000005: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:50:22.661 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:50:22.705 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '8D0DF282-17F6-4577-9213-B33E9C70903E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16890), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:50:22.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:22.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:22.865 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:22.865 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:50:22.865 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 16:50:23.036 4076 3051049642472 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:50:23.036 4076 3051049642472 MainThread INFO nova.compute.manager [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] VM Started (Lifecycle Event) 2021-09-10 16:50:23.036 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:23.036 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:23.036 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:50:23.036 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:23.084 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-ef8fff98-f6cf-4e98-9624-aea88762bd11 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:23.084 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.vmops [req-ef8fff98-f6cf-4e98-9624-aea88762bd11 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:23.098 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '8D0DF282-17F6-4577-9213-B33E9C70903E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=323799), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:50:23.098 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000005 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:50:23.098 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000005 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:50:23.098 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:23.098 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:23.119 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:23.159 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-ef8fff98-f6cf-4e98-9624-aea88762bd11 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Synchronizing instance power state after lifecycle event "Started"; current vm_state: error, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:50:23.205 4076 3051049642472 MainThread INFO nova.compute.manager [req-ef8fff98-f6cf-4e98-9624-aea88762bd11 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] During sync_power_state the instance has a pending task (rebuild_spawning). Skip. 2021-09-10 16:50:23.339 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:23.339 4076 3051049643352 MainThread DEBUG nova.objects.instance [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:50:23.486 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-1a42a634-2775-4513-8343-4c42e1dfbc62 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.141s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:23.556 4076 3051049642472 MainThread INFO nova.compute.manager [req-6ae9eb0a-1070-4ad6-9d9a-4949103b30c6 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:23.556 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-6ae9eb0a-1070-4ad6-9d9a-4949103b30c6 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:23.556 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-6ae9eb0a-1070-4ad6-9d9a-4949103b30c6 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:24.212 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:24.236 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:24.236 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:50:24.236 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:24.643 4076 3051049643704 MainThread INFO nova.compute.manager [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Rebuilding instance 2021-09-10 16:50:24.643 4076 3051049642472 MainThread INFO nova.compute.manager [req-4fc89878-eff3-486f-8622-fdc76203abb7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:24.643 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-4fc89878-eff3-486f-8622-fdc76203abb7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:24.675 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-4fc89878-eff3-486f-8622-fdc76203abb7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:24.768 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'trusted_certs' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:24.804 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:24.804 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:24.804 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:24.804 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:24.831 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:24.831 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:24.831 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:50:24.831 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:24.878 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:24.986 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'pci_requests' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:25.004 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'pci_devices' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:25.033 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'resources' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:25.065 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'migration_context' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:25.081 4076 3051049643704 MainThread INFO nova.virt.hyperv.serialconsoleops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Stopping instance instance-00000005 serial console handler. 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Performing Soft shutdown on instance _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:778 2021-09-10 16:50:25.081 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 60 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:25.113 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown failed: WMI job failed with status None. Error summary description: None. Error description: None Error code: 32768. _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:797 2021-09-10 16:50:25.174 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=7431MB free_disk=57GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:50:25.174 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 7 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:50:25.424 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=7680MB phys_disk=79GB used_disk=7GB total_vcpus=6 used_vcpus=7 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:50:25.456 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:25.505 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:25.757 4076 3051049642472 MainThread INFO nova.compute.manager [req-1e1bcd75-3303-431e-b61e-88125cb94f14 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:25.768 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-1e1bcd75-3303-431e-b61e-88125cb94f14 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:25.768 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-1e1bcd75-3303-431e-b61e-88125cb94f14 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:25.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:50:25.812 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:50:25.812 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:50:25.817 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.641s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:26.833 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:26.833 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:26.833 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:26.851 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:50:26.851 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:50:27.206 4076 3051049642472 MainThread INFO nova.compute.manager [req-468d299d-b705-42ef-a552-9101a9110437 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:27.206 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-468d299d-b705-42ef-a552-9101a9110437 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:27.206 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-468d299d-b705-42ef-a552-9101a9110437 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:28.521 4076 3051049642472 MainThread INFO nova.compute.manager [req-5fbce88a-5b66-4910-b0e3-0e7fdf50620c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:28.521 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-5fbce88a-5b66-4910-b0e3-0e7fdf50620c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:28.521 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-5fbce88a-5b66-4910-b0e3-0e7fdf50620c 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:28.632 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:28.664 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:50:28.788 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:28.808 4076 3051055215368 MainThread DEBUG nova.virt.hardware [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:50:28.808 4076 3051055215368 MainThread INFO nova.compute.claims [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Claim successful on node n-h1-808474-1 2021-09-10 16:50:28.972 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.hostops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:28.987 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.hostops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:29.224 4076 3051055215368 MainThread DEBUG nova.compute.provider_tree [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:50:29.268 4076 3051055215368 MainThread DEBUG nova.scheduler.client.report [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:50:29.268 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:29.268 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:50:29.378 4076 3051055215368 MainThread DEBUG nova.compute.utils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:50:29.378 4076 3051049642472 GreenThread-8 DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:50:29.378 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:50:29.414 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:50:29.616 4076 3051055217480 MainThread INFO nova.compute.manager [req-8b996c07-3f89-4f11-ac5b-4e50f43c3084 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:29.616 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-8b996c07-3f89-4f11-ac5b-4e50f43c3084 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:29.616 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-8b996c07-3f89-4f11-ac5b-4e50f43c3084 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:29.660 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:50:29.660 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Spawning new instance 2021-09-10 16:50:29.678 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "instance-0000000f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:29.678 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "instance-0000000f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:29.678 4076 3051055215368 MainThread DEBUG nova.block_device [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:29.678 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:29.678 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:29.690 4076 3051049642472 GreenThread-8 DEBUG nova.policy [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'acffa079bde44546a05ed304a8026d7c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b414c432778c4952ab38ce4062ccf6c8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:50:29.690 4076 3051055215368 MainThread DEBUG os_win.utils.pathutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Creating directory: C:\openstack\instances\instance-0000000f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:50:29.690 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000000f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:50:29.736 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Resizing VHD C:\openstack\instances\instance-0000000f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:50:30.814 4076 3051055217480 MainThread INFO nova.compute.manager [req-315028d3-f712-4cea-bbf9-cae7a939f137 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:30.814 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-315028d3-f712-4cea-bbf9-cae7a939f137 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:30.814 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-315028d3-f712-4cea-bbf9-cae7a939f137 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:31.408 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Successfully created port: c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:50:32.174 4076 3051055217480 MainThread INFO nova.compute.manager [req-9f9e2b3a-d61c-4d40-8ad9-8e8377e3002f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:32.174 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-9f9e2b3a-d61c-4d40-8ad9-8e8377e3002f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:32.174 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-9f9e2b3a-d61c-4d40-8ad9-8e8377e3002f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:32.892 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Successfully updated port: c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:50:32.911 4076 3051049642472 GreenThread-8 DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:32.911 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:50:33.017 4076 3051055217656 MainThread DEBUG nova.compute.manager [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Received event network-changed-c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:33.017 4076 3051055217656 MainThread DEBUG nova.compute.manager [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Refreshing instance network info cache due to event network-changed-c35b0100-fb8c-4ff3-9f5e-1faa360d72c4. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:50:33.068 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:50:33.486 4076 3051055217480 MainThread INFO nova.compute.manager [req-0d81030f-1797-4b17-943a-b97aaca079e8 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:33.486 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-0d81030f-1797-4b17-943a-b97aaca079e8 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:33.486 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-0d81030f-1797-4b17-943a-b97aaca079e8 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:34.149 4076 3051049642472 GreenThread-8 DEBUG nova.network.neutron [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:34.177 4076 3051049642472 GreenThread-8 DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:34.177 4076 3051049642472 GreenThread-8 DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Instance network_info: |[{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:50:34.177 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Preparing to wait for external event network-vif-plugged-c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:50:34.177 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:34.177 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:34.177 4076 3051055215368 MainThread DEBUG os_win.utils.compute.vmutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Creating VM instance-0000000f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:50:34.177 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:34.177 4076 3051055217656 MainThread DEBUG nova.network.neutron [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Refreshing network info cache for port c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:50:34.975 4076 3051055217480 MainThread INFO nova.compute.manager [req-85d07a25-e2d4-4084-bbc4-9956b1790806 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:34.975 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-85d07a25-e2d4-4084-bbc4-9956b1790806 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:34.975 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-85d07a25-e2d4-4084-bbc4-9956b1790806 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:35.020 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:50:35.114 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 50 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:35.146 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown failed: WMI job failed with status None. Error summary description: None. Error description: None Error code: 32768. _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:797 2021-09-10 16:50:35.714 4076 3051055217656 MainThread DEBUG nova.network.neutron [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updated VIF entry in instance network info cache for port c35b0100-fb8c-4ff3-9f5e-1faa360d72c4. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:50:35.714 4076 3051055217656 MainThread DEBUG nova.network.neutron [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:35.734 4076 3051055217656 MainThread DEBUG oslo_concurrency.lockutils [req-3ff2796f-1691-4ccd-a314-080516629318 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:36.284 4076 3051049642472 MainThread INFO nova.compute.manager [req-0f8ca70a-4337-4fce-8738-2754eee0b3bb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:36.284 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0f8ca70a-4337-4fce-8738-2754eee0b3bb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:36.304 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0f8ca70a-4337-4fce-8738-2754eee0b3bb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:37.424 4076 3051049642472 MainThread INFO nova.compute.manager [req-d4ea8a88-d79e-46f4-8e0e-452d43989ec1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:37.424 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-d4ea8a88-d79e-46f4-8e0e-452d43989ec1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:37.424 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-d4ea8a88-d79e-46f4-8e0e-452d43989ec1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:38.564 4076 3051049642472 MainThread INFO nova.compute.manager [req-8e4f7a19-25dc-4d57-aabf-97607c03ab10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:38.564 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8e4f7a19-25dc-4d57-aabf-97607c03ab10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:38.564 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-8e4f7a19-25dc-4d57-aabf-97607c03ab10 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:38.986 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-eafe4bcb-9655-47a3-ba17-2718c15d2fe0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Received event network-vif-plugged-c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:38.986 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-eafe4bcb-9655-47a3-ba17-2718c15d2fe0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:38.986 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-eafe4bcb-9655-47a3-ba17-2718c15d2fe0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:38.986 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-eafe4bcb-9655-47a3-ba17-2718c15d2fe0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Processing event network-vif-plugged-c35b0100-fb8c-4ff3-9f5e-1faa360d72c4 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:50:39.034 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Using config drive for instance 2021-09-10 16:50:39.306 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Creating config drive at C:\openstack\instances\instance-0000000f\configdrive.iso 2021-09-10 16:50:39.349 4076 3051055215368 MainThread DEBUG oslo_concurrency.processutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000000f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphuuk4gxn execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:50:39.692 4076 3051055217128 MainThread INFO nova.compute.manager [req-b7ea575c-a4cb-4003-9530-3454fbc38cd0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:39.692 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-b7ea575c-a4cb-4003-9530-3454fbc38cd0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:39.692 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-b7ea575c-a4cb-4003-9530-3454fbc38cd0 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:39.711 4076 3051055215368 MainThread DEBUG oslo_concurrency.processutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000000f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphuuk4gxn" returned: 0 in 0.360s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:50:40.034 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Setting boot order for instance: instance-0000000f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:50:40.096 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:50:40.129 4076 3051055215368 MainThread DEBUG os_win.utils.jobutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Waiting for WMI job: {'InstanceID': '7D1B5225-8131-4D0E-BEB5-C9DE2B425E7D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18842), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:50:40.427 4076 3051049642472 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:50:40.427 4076 3051049642472 MainThread INFO nova.compute.manager [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] VM Started (Lifecycle Event) 2021-09-10 16:50:40.427 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:40.427 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-0524caed-757b-42c8-8538-9c7e524c88d3 - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:40.427 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.vmops [req-0524caed-757b-42c8-8538-9c7e524c88d3 - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:40.461 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:40.489 4076 3051055215368 MainThread DEBUG os_win.utils.jobutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] WMI job succeeded: {'InstanceID': '7D1B5225-8131-4D0E-BEB5-C9DE2B425E7D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=275716), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:50:40.489 4076 3051055215368 MainThread DEBUG os_win.utils.compute.vmutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Successfully changed vm state of instance-0000000f to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:50:40.489 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Successfully changed state of VM instance-0000000f to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:50:40.489 4076 3051055215368 MainThread INFO nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Took 10.83 seconds to spawn the instance on the hypervisor. 2021-09-10 16:50:40.504 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:40.504 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:40.504 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-0524caed-757b-42c8-8538-9c7e524c88d3 - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:50:40.536 4076 3051049642472 MainThread INFO nova.compute.manager [req-0524caed-757b-42c8-8538-9c7e524c88d3 - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:50:40.661 4076 3051055215368 MainThread INFO nova.compute.manager [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Took 11.91 seconds to build instance. 2021-09-10 16:50:40.693 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-06d081e2-fffa-4782-95a8-5d91d01d2fa9 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 12.062s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:40.894 4076 3051049642472 MainThread INFO nova.compute.manager [req-c51bb493-0a56-45a2-b06a-0f68aecc29aa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:40.909 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c51bb493-0a56-45a2-b06a-0f68aecc29aa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:40.909 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-c51bb493-0a56-45a2-b06a-0f68aecc29aa 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:41.556 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:41.567 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:50:41.689 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:41.689 4076 3051055215368 MainThread DEBUG nova.virt.hardware [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:50:41.689 4076 3051055215368 MainThread INFO nova.compute.claims [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Claim successful on node n-h1-808474-1 2021-09-10 16:50:41.833 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.hostops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:41.865 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.hostops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:42.065 4076 3051055215368 MainThread DEBUG nova.compute.provider_tree [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:50:42.081 4076 3051055215368 MainThread DEBUG nova.scheduler.client.report [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:50:42.081 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:42.099 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:50:42.099 4076 3051049642472 MainThread INFO nova.compute.manager [req-937d0492-83c0-45a3-9eab-8322b88ca6eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:42.099 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-937d0492-83c0-45a3-9eab-8322b88ca6eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:42.099 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-937d0492-83c0-45a3-9eab-8322b88ca6eb 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:42.161 4076 3051055215368 MainThread DEBUG nova.compute.utils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:50:42.161 4076 3051049642472 GreenThread-9 DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:50:42.173 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:50:42.189 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:50:42.345 4076 3051049642472 GreenThread-9 DEBUG nova.policy [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'beffe9ba8529426fbb27e549254bd679', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91237ebda1b54d2dbc761e48fdd93516', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:50:42.393 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:50:42.409 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Spawning new instance 2021-09-10 16:50:42.409 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000012" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:42.409 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000012" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:42.409 4076 3051055215368 MainThread DEBUG nova.block_device [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:42.426 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:42.426 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:42.426 4076 3051055215368 MainThread DEBUG os_win.utils.pathutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating directory: C:\openstack\instances\instance-00000012 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:50:42.426 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000012\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:50:42.458 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Resizing VHD C:\openstack\instances\instance-00000012\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:50:42.894 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:42.894 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:42.894 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:42.894 4076 3051055217304 MainThread ERROR nova.compute.manager [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Traceback (most recent call last): 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] self._delete_instance(context, instance, bdms) 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] NameError: name 'instant' is not defined 2021-09-10 16:50:42.894 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] 2021-09-10 16:50:42.973 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:43.065 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:43.081 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Successfully created port: 43904798-2eab-4ec0-9b1b-f79a929eba99 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:50:43.097 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:43.124 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:43.299 4076 3051055217480 MainThread INFO nova.compute.manager [req-9f4bb412-365e-4851-9d27-8b817785fd6a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:43.299 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-9f4bb412-365e-4851-9d27-8b817785fd6a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:43.299 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-9f4bb412-365e-4851-9d27-8b817785fd6a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:43.331 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:50:43.331 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:50:43.331 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:43.331 4076 3051055217304 MainThread INFO nova.compute.manager [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:50:43.331 4076 3051055217304 MainThread ERROR oslo_messaging.rpc.server [req-15b22e7f-b2b4-415e-b41c-7824561917e6 acffa079bde44546a05ed304a8026d7c b414c432778c4952ab38ce4062ccf6c8 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:50:43.331 4076 3051055217304 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:50:43.829 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Successfully updated port: 43904798-2eab-4ec0-9b1b-f79a929eba99 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:50:43.848 4076 3051049642472 GreenThread-9 DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Acquired lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:43.848 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:50:43.897 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Received event network-changed-43904798-2eab-4ec0-9b1b-f79a929eba99 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:43.897 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Refreshing instance network info cache due to event network-changed-43904798-2eab-4ec0-9b1b-f79a929eba99. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:50:43.942 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:50:44.302 4076 3051049642472 GreenThread-9 DEBUG nova.network.neutron [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Updating instance_info_cache with network_info: [{"id": "43904798-2eab-4ec0-9b1b-f79a929eba99", "address": "fa:16:3e:9c:26:60", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap43904798-2e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:44.332 4076 3051049642472 GreenThread-9 DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Releasing lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:44.332 4076 3051049642472 GreenThread-9 DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Instance network_info: |[{"id": "43904798-2eab-4ec0-9b1b-f79a929eba99", "address": "fa:16:3e:9c:26:60", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap43904798-2e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:50:44.332 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Preparing to wait for external event network-vif-plugged-43904798-2eab-4ec0-9b1b-f79a929eba99 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:50:44.332 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:44.332 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:44.332 4076 3051055215368 MainThread DEBUG os_win.utils.compute.vmutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating VM instance-00000012 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:50:44.332 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:50:44.349 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Refreshing network info cache for port 43904798-2eab-4ec0-9b1b-f79a929eba99 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:50:44.736 4076 3051055217480 MainThread INFO nova.compute.manager [req-4ce37141-a6b7-41ab-8efd-0416ecdceab3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:44.736 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4ce37141-a6b7-41ab-8efd-0416ecdceab3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:44.736 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4ce37141-a6b7-41ab-8efd-0416ecdceab3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:45.144 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 40 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:45.193 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:50:45.439 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Updated VIF entry in instance network info cache for port 43904798-2eab-4ec0-9b1b-f79a929eba99. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:50:45.439 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Updating instance_info_cache with network_info: [{"id": "43904798-2eab-4ec0-9b1b-f79a929eba99", "address": "fa:16:3e:9c:26:60", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap43904798-2e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:50:45.495 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-ec09def6-328b-4c1f-bb95-e569aab31115 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:50:46.036 4076 3051049642472 MainThread INFO nova.compute.manager [req-2d8a93e2-6dca-4b64-8f6c-f0d1a84dd25d 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:46.036 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-2d8a93e2-6dca-4b64-8f6c-f0d1a84dd25d 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:46.036 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-2d8a93e2-6dca-4b64-8f6c-f0d1a84dd25d 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:47.236 4076 3051049642472 MainThread INFO nova.compute.manager [req-92b61da4-4528-42ad-9e10-b75f63886a43 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:47.236 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-92b61da4-4528-42ad-9e10-b75f63886a43 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:47.236 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-92b61da4-4528-42ad-9e10-b75f63886a43 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:48.549 4076 3051049642472 MainThread INFO nova.compute.manager [req-55a281f2-9f6d-4862-88ef-93db4f175cde 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:48.568 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-55a281f2-9f6d-4862-88ef-93db4f175cde 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:48.568 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-55a281f2-9f6d-4862-88ef-93db4f175cde 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:50.096 4076 3051049642472 MainThread INFO nova.compute.manager [req-fbef9d6f-de0a-43a5-a754-a7286bbf4834 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:50.096 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-fbef9d6f-de0a-43a5-a754-a7286bbf4834 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:50.096 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-fbef9d6f-de0a-43a5-a754-a7286bbf4834 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:50.408 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:50.408 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:50.408 4076 3051049642472 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:50:50.408 4076 3051049642472 MainThread INFO nova.compute.manager [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] VM Stopped (Lifecycle Event) 2021-09-10 16:50:50.439 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-efc7d140-8dd3-4ac6-8120-703eba46d1b8 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:50.439 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.vmops [req-efc7d140-8dd3-4ac6-8120-703eba46d1b8 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:50.488 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-efc7d140-8dd3-4ac6-8120-703eba46d1b8 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: active, current task_state: rebuilding, current DB power_state: 1, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:50:50.527 4076 3051049642472 MainThread INFO nova.compute.manager [req-efc7d140-8dd3-4ac6-8120-703eba46d1b8 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] During sync_power_state the instance has a pending task (rebuilding). Skip. 2021-09-10 16:50:51.322 4076 3051049642472 MainThread INFO nova.compute.manager [req-4c7f4216-b239-4c86-9503-aee021a58d49 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:51.345 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-4c7f4216-b239-4c86-9503-aee021a58d49 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:51.345 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-4c7f4216-b239-4c86-9503-aee021a58d49 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:51.393 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-2eeb3484-4dfa-4d8e-a6b8-1d28b4986e15 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Received event network-vif-plugged-43904798-2eab-4ec0-9b1b-f79a929eba99 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:50:51.393 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-2eeb3484-4dfa-4d8e-a6b8-1d28b4986e15 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:51.409 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-2eeb3484-4dfa-4d8e-a6b8-1d28b4986e15 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:51.409 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-2eeb3484-4dfa-4d8e-a6b8-1d28b4986e15 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Processing event network-vif-plugged-43904798-2eab-4ec0-9b1b-f79a929eba99 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:50:51.455 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Using config drive for instance 2021-09-10 16:50:51.705 4076 3051055215368 MainThread INFO nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Creating config drive at C:\openstack\instances\instance-00000012\configdrive.iso 2021-09-10 16:50:51.737 4076 3051055215368 MainThread DEBUG oslo_concurrency.processutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000012\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpo1ikoo_x execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:50:52.131 4076 3051055215368 MainThread DEBUG oslo_concurrency.processutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000012\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpo1ikoo_x" returned: 0 in 0.391s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:50:52.502 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Setting boot order for instance: instance-00000012: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:50:52.565 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:50:52.597 4076 3051055215368 MainThread DEBUG os_win.utils.jobutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '3A7F98F4-A584-4C76-A51D-C2091865296D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19040), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:50:52.614 4076 3051049642472 MainThread INFO nova.compute.manager [req-0a8e6087-6211-419b-9058-2a99469340fc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:52.614 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0a8e6087-6211-419b-9058-2a99469340fc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:52.614 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-0a8e6087-6211-419b-9058-2a99469340fc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:52.880 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000012" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:52.880 4076 3051049642472 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:50:52.880 4076 3051049642472 MainThread INFO nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] VM Started (Lifecycle Event) 2021-09-10 16:50:52.917 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-31311fba-6849-4e0a-8e04-b3858ccfce20 - - - - -] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:52.917 4076 3051049642472 MainThread DEBUG nova.virt.hyperv.vmops [req-31311fba-6849-4e0a-8e04-b3858ccfce20 - - - - -] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:52.926 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000012" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.046s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:52.954 4076 3051055215368 MainThread DEBUG os_win.utils.jobutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job succeeded: {'InstanceID': '3A7F98F4-A584-4C76-A51D-C2091865296D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=278533), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:50:52.954 4076 3051055215368 MainThread DEBUG os_win.utils.compute.vmutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed vm state of instance-00000012 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:50:52.954 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed state of VM instance-00000012 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:50:52.954 4076 3051055215368 MainThread INFO nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Took 10.56 seconds to spawn the instance on the hypervisor. 2021-09-10 16:50:52.954 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:50:52.954 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.vmops [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:50:52.971 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-31311fba-6849-4e0a-8e04-b3858ccfce20 - - - - -] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:50:53.004 4076 3051049642472 MainThread INFO nova.compute.manager [req-31311fba-6849-4e0a-8e04-b3858ccfce20 - - - - -] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:50:53.110 4076 3051055215368 MainThread INFO nova.compute.manager [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Took 11.45 seconds to build instance. 2021-09-10 16:50:53.130 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-684e4182-9e0d-4909-aea0-820a110b345c beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 11.579s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:53.845 4076 3051049642472 MainThread INFO nova.compute.manager [req-dfe3a792-7250-4a6e-8434-e72b0c5dfbb1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:53.845 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-dfe3a792-7250-4a6e-8434-e72b0c5dfbb1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:53.845 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-dfe3a792-7250-4a6e-8434-e72b0c5dfbb1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:54.970 4076 3051049642472 MainThread INFO nova.compute.manager [req-7c1c4793-176a-4a48-9b4c-04689fb5f8e2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:54.970 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-7c1c4793-176a-4a48-9b4c-04689fb5f8e2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:54.970 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-7c1c4793-176a-4a48-9b4c-04689fb5f8e2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:55.151 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown instance, timeout remaining: 30 _soft_shutdown c:\openstack\build\nova\nova\virt\hyperv\vmops.py:788 2021-09-10 16:50:55.192 4076 3051049643704 MainThread INFO nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Soft shutdown succeeded. 2021-09-10 16:50:55.192 4076 3051049643704 MainThread INFO nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Got request to destroy instance 2021-09-10 16:50:55.234 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:55.256 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:50:55.481 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:55.481 4076 3051055217480 MainThread DEBUG nova.virt.hardware [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:50:55.481 4076 3051055217480 MainThread INFO nova.compute.claims [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Claim successful on node n-h1-808474-1 2021-09-10 16:50:55.679 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:50:55.697 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:50:55.925 4076 3051055217480 MainThread DEBUG nova.compute.provider_tree [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:50:55.943 4076 3051055217480 MainThread DEBUG nova.scheduler.client.report [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:50:55.943 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:55.958 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "d5c286bc-3fa1-47d5-96ca-aab4277f06c9" acquired by "nova.compute.manager.ComputeManager._validate_instance_group_policy.._do_validation" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:55.958 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] No node specified, defaulting to n-h1-808474-1 _get_nodename c:\openstack\build\nova\nova\compute\manager.py:9984 2021-09-10 16:50:56.005 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "d5c286bc-3fa1-47d5-96ca-aab4277f06c9" released by "nova.compute.manager.ComputeManager._validate_instance_group_policy.._do_validation" :: held 0.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:56.005 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:50:56.106 4076 3051055217480 MainThread DEBUG nova.compute.utils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:50:56.106 4076 3051049642472 GreenThread-10 DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:50:56.106 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:50:56.160 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:50:56.379 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:50:56.392 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:56.392 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:56.392 4076 3051049643704 MainThread DEBUG os_win.utils.compute.vmutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000005 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:50:56.392 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000005 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:50:56.455 4076 3051049642472 GreenThread-10 DEBUG nova.policy [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bf4dce0601aa49c0b9ea6c48f2953557', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ae864ac350142778dcbcb12a8b3800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:50:56.455 4076 3051049643704 MainThread DEBUG os_win.utils.jobutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '2CFEF721-9AEC-43FC-84C6-0DA138302FF8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13757), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:50:56.536 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:50:56.536 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Spawning new instance 2021-09-10 16:50:56.552 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-00000014" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:56.552 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-00000014" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:56.552 4076 3051055217480 MainThread DEBUG nova.block_device [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:56.552 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:56.552 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:56.565 4076 3051055217480 MainThread DEBUG os_win.utils.pathutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating directory: C:\openstack\instances\instance-00000014 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:50:56.565 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000014\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:50:56.581 4076 3051049643704 MainThread DEBUG os_win.utils.jobutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '2CFEF721-9AEC-43FC-84C6-0DA138302FF8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=39258), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:50:56.581 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:56.581 4076 3051049643704 MainThread DEBUG os_win.utils.pathutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Removing directory: C:\openstack\instances\instance-00000005 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:50:56.581 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:56.602 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Resizing VHD C:\openstack\instances\instance-00000014\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:50:56.645 4076 3051055215368 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-52694233-7879-466e-bd52-bb83f831564f ca2efda67fb64954b0bfece0cc597f88 398a3c674d0e41c1a7e4459b18602bb9 - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] check_can_live_migrate_source called check_can_live_migrate_source c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:153 2021-09-10 16:50:56.645 4076 3051055215368 MainThread DEBUG nova.compute.manager [req-52694233-7879-466e-bd52-bb83f831564f ca2efda67fb64954b0bfece0cc597f88 398a3c674d0e41c1a7e4459b18602bb9 - default default] source check data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids=,vifs=,wait_for_vif_plugged=) check_can_live_migrate_source c:\openstack\build\nova\nova\compute\manager.py:8151 2021-09-10 16:50:57.018 4076 3051049643704 MainThread INFO nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Spawning new instance 2021-09-10 16:50:57.018 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:57.018 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:57.018 4076 3051049643704 MainThread DEBUG nova.block_device [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:50:57.018 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:57.018 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:57.053 4076 3051049643704 MainThread DEBUG os_win.utils.pathutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating directory: C:\openstack\instances\instance-00000005 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:50:57.053 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000005\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:50:57.067 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Resizing VHD C:\openstack\instances\instance-00000005\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:50:57.098 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Preparing to wait for external event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:50:57.098 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:57.098 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:57.098 4076 3051049643704 MainThread DEBUG os_win.utils.compute.vmutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Creating VM instance-00000005 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:50:57.939 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:50:58.080 4076 3051055215368 MainThread INFO nova.compute.manager [req-4a840c34-e2dc-4f0b-af32-ba2927baa630 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:58.080 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-4a840c34-e2dc-4f0b-af32-ba2927baa630 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:58.080 4076 3051055215368 MainThread DEBUG oslo_concurrency.lockutils [req-4a840c34-e2dc-4f0b-af32-ba2927baa630 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:50:58.472 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Successfully created port: 799c7188-eef9-4b77-8d75-2cfc73d1a87e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:50:59.676 4076 3051055214664 MainThread INFO nova.compute.manager [req-89f00003-8232-41ae-bebd-50ec2b9bb27e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:50:59.676 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-89f00003-8232-41ae-bebd-50ec2b9bb27e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:50:59.676 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-89f00003-8232-41ae-bebd-50ec2b9bb27e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:00.435 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Successfully updated port: 799c7188-eef9-4b77-8d75-2cfc73d1a87e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:00.479 4076 3051049642472 GreenThread-10 DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:00.479 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:00.598 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:00.783 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] check_can_live_migrate_destination called check_can_live_migrate_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:132 2021-09-10 16:51:00.783 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.pathutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Checking if C:\openstack\instances and \\n-h2-808474-1\C$\openstack\instances point to the same location. check_dirs_shared_storage c:\openstack\build\nova\nova\virt\hyperv\pathutils.py:183 2021-09-10 16:51:00.802 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] destination check data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=,migration=,old_vol_attachment_ids=,vifs=,wait_for_vif_plugged=) check_can_live_migrate_destination c:\openstack\build\nova\nova\compute\manager.py:8040 2021-09-10 16:51:01.017 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] cleanup_live_migration_destination_check called cleanup_live_migration_destination_check c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:148 2021-09-10 16:51:01.736 4076 3051049642472 GreenThread-10 DEBUG nova.network.neutron [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:01.769 4076 3051049642472 GreenThread-10 DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:01.769 4076 3051049642472 GreenThread-10 DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Instance network_info: |[{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:01.769 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Preparing to wait for external event network-vif-plugged-799c7188-eef9-4b77-8d75-2cfc73d1a87e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:01.769 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:01.769 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:01.769 4076 3051055217480 MainThread DEBUG os_win.utils.compute.vmutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating VM instance-00000014 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:01.786 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Received event network-changed-799c7188-eef9-4b77-8d75-2cfc73d1a87e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:01.786 4076 3051049642472 MainThread DEBUG nova.compute.manager [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Refreshing instance network info cache due to event network-changed-799c7188-eef9-4b77-8d75-2cfc73d1a87e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:01.786 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:01.786 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Refreshing network info cache for port 799c7188-eef9-4b77-8d75-2cfc73d1a87e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:02.310 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Took 4.27 seconds for pre_live_migration on destination host n-h2-808474-1. 2021-09-10 16:51:02.335 4076 3051055217304 MainThread DEBUG nova.compute.manager [-] live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=Migration(6b31a29f-d5d6-433f-bef2-453ac047b43e),old_vol_attachment_ids={},vifs=[VIFMigrateData],wait_for_vif_plugged=True) _do_live_migration c:\openstack\build\nova\nova\compute\manager.py:8499 2021-09-10 16:51:02.335 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.livemigrationops [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] live_migration called live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:52 2021-09-10 16:51:02.335 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000012" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:02.335 4076 3051055217304 MainThread INFO nova.virt.hyperv.serialconsoleops [-] Stopping instance instance-00000012 serial console handler. 2021-09-10 16:51:02.335 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000012" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:02.335 4076 3051055217304 MainThread DEBUG os_win.utils.pathutils [-] Creating directory: \\n-h2-808474-1\C$\openstack\instances\instance-00000012 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:02.367 4076 3051055216952 MainThread INFO nova.compute.manager [req-2eb55cb3-1d8d-4614-b499-05055ede50f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:02.367 4076 3051055216952 MainThread DEBUG oslo_concurrency.lockutils [req-2eb55cb3-1d8d-4614-b499-05055ede50f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:02.367 4076 3051055216952 MainThread DEBUG oslo_concurrency.lockutils [req-2eb55cb3-1d8d-4614-b499-05055ede50f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:02.551 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-2a476dc9-857d-48bd-98e2-82dd809b99dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:02.551 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-2a476dc9-857d-48bd-98e2-82dd809b99dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:02.551 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-2a476dc9-857d-48bd-98e2-82dd809b99dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:02.551 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-2a476dc9-857d-48bd-98e2-82dd809b99dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] No event matching network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 in dict_keys([('network-vif-plugged', '298cc953-6875-4186-9b30-11fa49791716')]) pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:330 2021-09-10 16:51:02.551 4076 3051055217128 MainThread WARNING nova.compute.manager [req-2a476dc9-857d-48bd-98e2-82dd809b99dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received unexpected event network-vif-unplugged-298cc953-6875-4186-9b30-11fa49791716 for instance with vm_state active and task_state rebuild_spawning. 2021-09-10 16:51:02.619 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-12b6bf74-a2a9-4cd7-9024-67c8f5ccecb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Received event network-vif-unplugged-43904798-2eab-4ec0-9b1b-f79a929eba99 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:02.628 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-12b6bf74-a2a9-4cd7-9024-67c8f5ccecb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:02.628 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-12b6bf74-a2a9-4cd7-9024-67c8f5ccecb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:02.628 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-12b6bf74-a2a9-4cd7-9024-67c8f5ccecb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] No waiting events found dispatching network-vif-unplugged-43904798-2eab-4ec0-9b1b-f79a929eba99 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:51:02.628 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-12b6bf74-a2a9-4cd7-9024-67c8f5ccecb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Received event network-vif-unplugged-43904798-2eab-4ec0-9b1b-f79a929eba99 for instance with task_state migrating. _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10284 2021-09-10 16:51:02.642 4076 3051055217304 MainThread DEBUG os_win.utils.compute.livemigrationutils [-] Getting live migration networks for host: n-h2-808474-1 _get_ip_address_list c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:172 2021-09-10 16:51:02.691 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids=,vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8168 2021-09-10 16:51:02.691 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:02.722 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:02.737 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:02.814 4076 3051055217304 MainThread DEBUG os_win.utils.compute.livemigrationutils [-] Starting live migration for VM: instance-00000012 _live_migrate_vm c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:162 2021-09-10 16:51:02.877 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [-] Waiting for WMI job: {'InstanceID': '971E083E-2FFC-4E7C-9536-618FBBD11159', 'Description': 'Moving Virtual Machine and Storage', 'ElementName': 'Moving Virtual Machine and Storage', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23387), 'Cancellable': True, 'JobType': 306, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:02.963 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updated VIF entry in instance network info cache for port 799c7188-eef9-4b77-8d75-2cfc73d1a87e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:02.963 4076 3051049642472 MainThread DEBUG nova.network.neutron [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:02.987 4076 3051049642472 MainThread DEBUG oslo_concurrency.lockutils [req-f20171b3-90b3-4123-a071-f2be05e0d105 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:03.566 4076 3051055218184 MainThread INFO nova.compute.manager [req-ff6f703a-8f2e-4556-b9ee-b0c74ac7a379 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:03.566 4076 3051055218184 MainThread DEBUG oslo_concurrency.lockutils [req-ff6f703a-8f2e-4556-b9ee-b0c74ac7a379 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:03.566 4076 3051055218184 MainThread DEBUG oslo_concurrency.lockutils [req-ff6f703a-8f2e-4556-b9ee-b0c74ac7a379 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:03.910 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:03.950 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:03.950 4076 3051055217128 MainThread DEBUG os_brick.utils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:03.950 4076 3051055217128 MainThread DEBUG os_brick.utils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:05.376 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Received event network-changed-43904798-2eab-4ec0-9b1b-f79a929eba99 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:05.376 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Refreshing instance network info cache due to event network-changed-43904798-2eab-4ec0-9b1b-f79a929eba99. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:05.376 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:05.376 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Refreshing network info cache for port 43904798-2eab-4ec0-9b1b-f79a929eba99 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:05.393 4076 3051093168728 MainThread INFO nova.compute.manager [req-07bfe02b-a56b-493b-9407-142e3ce850ce 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:05.393 4076 3051093168728 MainThread DEBUG oslo_concurrency.lockutils [req-07bfe02b-a56b-493b-9407-142e3ce850ce 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:05.393 4076 3051093168728 MainThread DEBUG oslo_concurrency.lockutils [req-07bfe02b-a56b-493b-9407-142e3ce850ce 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:06.259 4076 3051093168728 MainThread DEBUG nova.compute.manager [req-d9ccb0de-80a0-48ed-ac29-4169a771c5eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Received event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:06.259 4076 3051093168728 MainThread DEBUG oslo_concurrency.lockutils [req-d9ccb0de-80a0-48ed-ac29-4169a771c5eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:06.259 4076 3051093168728 MainThread DEBUG oslo_concurrency.lockutils [req-d9ccb0de-80a0-48ed-ac29-4169a771c5eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:06.259 4076 3051093168728 MainThread DEBUG nova.compute.manager [req-d9ccb0de-80a0-48ed-ac29-4169a771c5eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Processing event network-vif-plugged-298cc953-6875-4186-9b30-11fa49791716 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:06.302 4076 3051049643704 MainThread INFO nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Using config drive for instance 2021-09-10 16:51:06.332 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'ec2_ids' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:51:06.348 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'keypairs' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:51:06.376 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lazy-loading 'device_metadata' on Instance uuid f830cec8-b836-4470-83fd-f48ca352a795 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:51:06.612 4076 3051049643704 MainThread INFO nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Creating config drive at C:\openstack\instances\instance-00000005\configdrive.iso 2021-09-10 16:51:06.657 4076 3051049643704 MainThread DEBUG oslo_concurrency.processutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8bdxtwmg execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:06.979 4076 3051049643704 MainThread DEBUG oslo_concurrency.processutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000005\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8bdxtwmg" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:07.096 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Updated VIF entry in instance network info cache for port 43904798-2eab-4ec0-9b1b-f79a929eba99. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:07.096 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Updating instance_info_cache with network_info: [{"id": "43904798-2eab-4ec0-9b1b-f79a929eba99", "address": "fa:16:3e:9c:26:60", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap43904798-2e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {"migrating_to": "n-h2-808474-1"}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:07.145 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-141d29fe-5047-4c0b-983c-db025a0561f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-359fd482-69f5-4194-9466-f46ea31258d3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:07.166 4076 3051093168552 MainThread INFO nova.compute.manager [req-babab777-f6e1-4b15-882a-858603e3118f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:07.178 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-babab777-f6e1-4b15-882a-858603e3118f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:07.178 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-babab777-f6e1-4b15-882a-858603e3118f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:07.392 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Setting boot order for instance: instance-00000005: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:07.455 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:07.494 4076 3051049643704 MainThread DEBUG os_win.utils.jobutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Waiting for WMI job: {'InstanceID': '248DBD24-20DE-4490-9098-15D37586FCB9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22725), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:07.807 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:07.807 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] VM Started (Lifecycle Event) 2021-09-10 16:51:07.817 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:07.854 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c7525220-5914-4fdf-b1c5-298af23f0c95 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:07.854 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-c7525220-5914-4fdf-b1c5-298af23f0c95 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:07.854 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000005" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:07.872 4076 3051049643704 MainThread DEBUG os_win.utils.jobutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] WMI job succeeded: {'InstanceID': '248DBD24-20DE-4490-9098-15D37586FCB9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=310656), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:07.872 4076 3051049643704 MainThread DEBUG os_win.utils.compute.vmutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed vm state of instance-00000005 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:51:07.872 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Successfully changed state of VM instance-00000005 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:51:07.891 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:07.900 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:07.925 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [-] Waiting for WMI job: {'InstanceID': '971E083E-2FFC-4E7C-9536-618FBBD11159', 'Description': 'Moving Virtual Machine and Storage', 'ElementName': 'Moving Virtual Machine and Storage', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(seconds=5, microseconds=63719), 'Cancellable': True, 'JobType': 306, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 19}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:07.925 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c7525220-5914-4fdf-b1c5-298af23f0c95 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: rebuild_spawning, current DB power_state: 1, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:07.957 4076 3051093168552 MainThread INFO nova.compute.manager [req-c7525220-5914-4fdf-b1c5-298af23f0c95 - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] During sync_power_state the instance has a pending task (rebuild_spawning). Skip. 2021-09-10 16:51:08.069 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:08.069 4076 3051049643704 MainThread DEBUG nova.objects.instance [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:51:08.144 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-569d1a23-c94b-475f-93a2-d8b062f0b443 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:08.221 4076 3051093168552 MainThread INFO nova.compute.manager [req-73ab27ac-01f4-4cc5-9d35-71a9b21a1c5e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:08.221 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-73ab27ac-01f4-4cc5-9d35-71a9b21a1c5e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:08.251 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-73ab27ac-01f4-4cc5-9d35-71a9b21a1c5e 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:09.627 4076 3051093168552 MainThread INFO nova.compute.manager [req-c6cc99bd-68ab-4a66-a634-7b2def2afd87 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:09.627 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-c6cc99bd-68ab-4a66-a634-7b2def2afd87 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:09.627 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-c6cc99bd-68ab-4a66-a634-7b2def2afd87 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:09.704 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-fa11efbb-c7b8-4bdc-89da-071d5fbac8aa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Received event network-vif-plugged-799c7188-eef9-4b77-8d75-2cfc73d1a87e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:09.704 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-fa11efbb-c7b8-4bdc-89da-071d5fbac8aa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:09.704 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-fa11efbb-c7b8-4bdc-89da-071d5fbac8aa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:09.704 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-fa11efbb-c7b8-4bdc-89da-071d5fbac8aa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Processing event network-vif-plugged-799c7188-eef9-4b77-8d75-2cfc73d1a87e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:09.743 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Using config drive for instance 2021-09-10 16:51:09.957 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Creating config drive at C:\openstack\instances\instance-00000014\configdrive.iso 2021-09-10 16:51:10.001 4076 3051055217480 MainThread DEBUG oslo_concurrency.processutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000014\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpb239dmsv execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:10.273 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:10.273 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:10.273 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:10.273 4076 3051093168904 MainThread ERROR nova.compute.manager [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Traceback (most recent call last): 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] self._delete_instance(context, instance, bdms) 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] NameError: name 'instant' is not defined 2021-09-10 16:51:10.273 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] 2021-09-10 16:51:10.345 4076 3051055217480 MainThread DEBUG oslo_concurrency.processutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000014\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpb239dmsv" returned: 0 in 0.343s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:10.376 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.110s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:10.583 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:10.611 4076 3051093168904 MainThread DEBUG nova.virt.hyperv.hostops [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:10.642 4076 3051093168904 MainThread DEBUG nova.virt.hyperv.hostops [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:10.688 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Setting boot order for instance: instance-00000014: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:10.755 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:10.766 4076 3051093168200 MainThread INFO nova.compute.manager [req-0d987ed5-2750-42c8-8b83-c9757063c88a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:10.766 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-0d987ed5-2750-42c8-8b83-c9757063c88a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:10.766 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-0d987ed5-2750-42c8-8b83-c9757063c88a 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:10.799 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'A4D03C4B-D0B4-4BE8-B1BD-0DEB60F09533', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16630), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:10.908 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] pre_live_migration called pre_live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:92 2021-09-10 16:51:10.943 4076 3051093168904 MainThread DEBUG nova.compute.provider_tree [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:10.943 4076 3051055217128 MainThread DEBUG nova.block_device [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] block_device_list ['sdb'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:10.943 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:10.943 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:10.943 4076 3051055217128 MainThread DEBUG os_brick.initiator.connector [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Factory for iscsi on None factory c:\python37\lib\site-packages\os_brick\initiator\connector.py:282 2021-09-10 16:51:10.943 4076 3051055217128 MainThread INFO os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] No iSCSI initiator was explicitly requested. The Microsoft iSCSI initiator will choose the initiator when establishing sessions. 2021-09-10 16:51:10.943 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99', 'target_lun': 0, 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'auth_method': 'CHAP', 'auth_username': '64HsfuxbJfZXqwRTayVC', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:10.943 4076 3051055217128 MainThread INFO os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:51:10.961 4076 3051055217128 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:51:10.985 4076 3051093168904 MainThread DEBUG nova.scheduler.client.report [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:10.985 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:11.001 4076 3051093168904 MainThread INFO nova.compute.manager [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:51:11.001 4076 3051093168904 MainThread ERROR oslo_messaging.rpc.server [req-0539013e-cdf1-438a-840e-df008460a4cc 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:51:11.001 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:51:11.133 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000014" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:11.133 4076 3051093168904 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:11.133 4076 3051093168904 MainThread INFO nova.compute.manager [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] VM Started (Lifecycle Event) 2021-09-10 16:51:11.158 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-05c801f7-af47-49be-8776-70e728e65331 - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:11.158 4076 3051093168904 MainThread DEBUG nova.virt.hyperv.vmops [req-05c801f7-af47-49be-8776-70e728e65331 - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:11.174 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000014" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.046s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:11.204 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job succeeded: {'InstanceID': 'A4D03C4B-D0B4-4BE8-B1BD-0DEB60F09533', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=314974), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:11.204 4076 3051055217480 MainThread DEBUG os_win.utils.compute.vmutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed vm state of instance-00000014 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:51:11.204 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed state of VM instance-00000014 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:51:11.204 4076 3051055217480 MainThread INFO nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Took 14.67 seconds to spawn the instance on the hypervisor. 2021-09-10 16:51:11.204 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:11.204 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:11.235 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-05c801f7-af47-49be-8776-70e728e65331 - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:11.254 4076 3051093168904 MainThread INFO nova.compute.manager [req-05c801f7-af47-49be-8776-70e728e65331 - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:11.330 4076 3051055217480 MainThread INFO nova.compute.manager [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Took 15.89 seconds to build instance. 2021-09-10 16:51:11.361 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42aedf44-51b4-4cd9-9080-084a8d568de0 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 16.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:11.485 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [-] WMI job succeeded: {'InstanceID': '971E083E-2FFC-4E7C-9536-618FBBD11159', 'Description': 'Moving Virtual Machine and Storage', 'ElementName': 'Moving Virtual Machine and Storage', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(seconds=8, microseconds=559986), 'Cancellable': True, 'JobType': 306, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:11.485 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.livemigrationops [-] Calling live migration post_method for instance: instance-00000012 live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:86 2021-09-10 16:51:11.485 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] _post_live_migration() is started.. 2021-09-10 16:51:11.485 4076 3051055217304 MainThread DEBUG os_win.utils.pathutils [-] Removing directory: C:\openstack\instances\instance-00000012 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:51:11.508 4076 3051055217304 MainThread DEBUG os_brick.utils [-] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:11.508 4076 3051055217304 MainThread DEBUG os_brick.utils [-] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:12.072 4076 3051093168904 MainThread INFO nova.compute.manager [req-5c787c41-1b31-4e84-885a-4517cfa8a879 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:12.085 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-5c787c41-1b31-4e84-885a-4517cfa8a879 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:12.085 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-5c787c41-1b31-4e84-885a-4517cfa8a879 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:12.269 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:12.269 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:12.269 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:12.269 4076 3051093168904 MainThread ERROR nova.compute.manager [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Traceback (most recent call last): 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] self._delete_instance(context, instance, bdms) 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] NameError: name 'instant' is not defined 2021-09-10 16:51:12.269 4076 3051093168904 MainThread TRACE nova.compute.manager [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] 2021-09-10 16:51:12.365 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.094s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG nova.network.neutron [-] Activated binding for port 43904798-2eab-4ec0-9b1b-f79a929eba99 and host n-h2-808474-1 migrate_instance_start c:\openstack\build\nova\nova\network\neutron.py:3012 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Calling driver.post_live_migration_at_source with original source VIFs from migrate_data: [{"id": "43904798-2eab-4ec0-9b1b-f79a929eba99", "address": "fa:16:3e:9c:26:60", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap43904798-2e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] _post_live_migration c:\openstack\build\nova\nova\compute\manager.py:8815 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Hypervisor driver does not support post_live_migration_at_source method _post_live_migration c:\openstack\build\nova\nova\compute\manager.py:8819 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.free_pci_device_allocations_for_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.free_pci_device_allocations_for_instance" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:12.423 4076 3051055217304 MainThread DEBUG nova.objects.instance [-] Lazy-loading 'migration_context' on Instance uuid 359fd482-69f5-4194-9466-f46ea31258d3 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:51:12.439 4076 3051055217304 MainThread DEBUG nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Calling driver.cleanup from _post_live_migration _post_live_migration c:\openstack\build\nova\nova\compute\manager.py:8837 2021-09-10 16:51:12.548 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:12.582 4076 3051093168904 MainThread DEBUG nova.virt.hyperv.hostops [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:12.614 4076 3051093168904 MainThread DEBUG nova.virt.hyperv.hostops [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:12.839 4076 3051093168904 MainThread DEBUG nova.compute.provider_tree [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:12.862 4076 3051093168904 MainThread DEBUG nova.scheduler.client.report [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:12.862 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:12.862 4076 3051093168904 MainThread INFO nova.compute.manager [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:51:12.862 4076 3051093168904 MainThread ERROR oslo_messaging.rpc.server [req-b8bdc918-d925-41d7-a406-ab6dfd90b1a9 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:51:12.862 4076 3051093168904 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:51:13.189 4076 3051055217480 MainThread INFO nova.compute.manager [req-a6e0adad-438b-4392-a16e-31ac69c09ecc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:13.189 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-a6e0adad-438b-4392-a16e-31ac69c09ecc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:13.222 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-a6e0adad-438b-4392-a16e-31ac69c09ecc 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:13.924 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:13.924 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "359fd482-69f5-4194-9466-f46ea31258d3-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:13.944 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:13.944 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:13.944 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:51:13.944 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [-] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:13.973 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [-] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:14.223 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=5310MB free_disk=54GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:51:14.223 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:14.303 4076 3051055217480 MainThread INFO nova.compute.manager [req-18dc26b7-eaf4-45e3-8af1-4d15a75f09a1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:14.303 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-18dc26b7-eaf4-45e3-8af1-4d15a75f09a1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:14.303 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-18dc26b7-eaf4-45e3-8af1-4d15a75f09a1 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:14.315 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Migration for instance 359fd482-69f5-4194-9466-f46ea31258d3 refers to another host's instance! _pair_instances_to_migrations c:\openstack\build\nova\nova\compute\resource_tracker.py:911 2021-09-10 16:51:14.315 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Migration for instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 refers to another host's instance! _pair_instances_to_migrations c:\openstack\build\nova\nova\compute\resource_tracker.py:911 2021-09-10 16:51:14.355 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Skipping migration as instance is neither resizing nor live-migrating. _update_usage_from_migrations c:\openstack\build\nova\nova\compute\resource_tracker.py:1390 2021-09-10 16:51:14.355 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:14.355 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:14.355 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:14.376 4076 3051055217480 MainThread ERROR nova.compute.manager [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] Traceback (most recent call last): 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] self._delete_instance(context, instance, bdms) 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] NameError: name 'instant' is not defined 2021-09-10 16:51:14.376 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f830cec8-b836-4470-83fd-f48ca352a795] 2021-09-10 16:51:14.411 4076 3051055217304 MainThread INFO nova.compute.resource_tracker [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating resource usage from migration 6caf2594-4ef9-4a93-9694-0c999f81f712 2021-09-10 16:51:14.411 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Starting to track incoming migration 6caf2594-4ef9-4a93-9694-0c999f81f712 with flavor 452 _update_usage_from_migration c:\openstack\build\nova\nova\compute\resource_tracker.py:1330 2021-09-10 16:51:14.471 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.109s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:14.501 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.501 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.501 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Migration 6b31a29f-d5d6-433f-bef2-453ac047b43e is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1537 2021-09-10 16:51:14.501 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.501 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread WARNING nova.compute.resource_tracker [-] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 has been moved to another host n-h2-808474-1(n-h2-808474-1). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Total usable vcpus: 6, total allocated vcpus: 10 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:51:14.564 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=10752MB phys_disk=79GB used_disk=10GB total_vcpus=6 used_vcpus=10 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:51:14.610 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [-] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:14.629 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [-] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:14.896 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [-] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:14.931 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [-] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:14.931 4076 3051055217304 MainThread DEBUG nova.compute.resource_tracker [-] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:51:14.931 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:14.941 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.281s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:14.960 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 359fd482-69f5-4194-9466-f46ea31258d3] Migrating instance to n-h2-808474-1 finished successfully. 2021-09-10 16:51:15.004 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:15.019 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:15.128 4076 3051055217304 MainThread INFO nova.scheduler.client.report [-] Deleted allocation for migration 6b31a29f-d5d6-433f-bef2-453ac047b43e 2021-09-10 16:51:15.299 4076 3051055217480 MainThread DEBUG nova.compute.provider_tree [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:15.316 4076 3051055217480 MainThread DEBUG nova.scheduler.client.report [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:15.316 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:15.316 4076 3051055217480 MainThread INFO nova.compute.manager [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:51:15.316 4076 3051055217480 MainThread ERROR oslo_messaging.rpc.server [req-42944892-ab52-46ca-b056-37521e5c95c8 832a42f6b78c40c0bee881dab45d7c65 fa9a142ec08846a38fa7f3cbcd2b53d8 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:51:15.316 4076 3051055217480 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:51:15.644 4076 3051055217480 MainThread INFO nova.compute.manager [req-04129187-fd29-4192-8f4f-d134021145a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:15.644 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-04129187-fd29-4192-8f4f-d134021145a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:15.644 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-04129187-fd29-4192-8f4f-d134021145a2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:16.002 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:51:16.376 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:51:16.798 4076 3051055217480 MainThread INFO nova.compute.manager [req-8b46675e-9615-42d2-8bab-32c8fd27d8f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:16.798 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-8b46675e-9615-42d2-8bab-32c8fd27d8f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:16.798 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-8b46675e-9615-42d2-8bab-32c8fd27d8f9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:17.111 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:17.159 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:51:17.235 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] <== connect_volume: return (6292ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '360014056D06827A0A37453798A12FB65'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:17.235 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] ==> get_volume_paths: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99', 'target_lun': 0, 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'auth_method': 'CHAP', 'auth_username': '64HsfuxbJfZXqwRTayVC', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:17.235 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] <== get_volume_paths: return (0ms) ['\\\\.\\PhysicalDrive1'] trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:17.329 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:17.329 4076 3051055217304 MainThread DEBUG nova.virt.hardware [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:51:17.329 4076 3051055217304 MainThread INFO nova.compute.claims [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Claim successful on node n-h1-808474-1 2021-09-10 16:51:17.487 4076 3051055217128 MainThread DEBUG os_win.utils.compute.livemigrationutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Getting live migration networks for host: n-h1-808474-1 _get_ip_address_list c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:172 2021-09-10 16:51:17.533 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:17.565 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:17.799 4076 3051055217128 MainThread DEBUG os_win.utils.compute.livemigrationutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Creating planned VM for VM: instance-0000000e _create_planned_vm c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:87 2021-09-10 16:51:17.845 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:17.866 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:17.866 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:17.866 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:51:18.003 4076 3051055217304 MainThread DEBUG nova.compute.utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:18.003 4076 3051055217480 GreenThread-11 DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:51:18.003 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:51:18.034 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:51:18.099 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Waiting for WMI job: {'InstanceID': '44573840-6F6D-473E-A69A-839D25E852F1', 'Description': 'Creating Remote Virtual Machine', 'ElementName': 'Creating Remote Virtual Machine', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=122795), 'Cancellable': True, 'JobType': 300, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 3}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:18.301 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:51:18.315 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Spawning new instance 2021-09-10 16:51:18.333 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:18.333 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:18.333 4076 3051055217304 MainThread DEBUG nova.block_device [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:18.333 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:18.333 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:18.355 4076 3051055217304 MainThread DEBUG os_win.utils.pathutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating directory: C:\openstack\instances\instance-00000016 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:18.355 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000016\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:51:18.391 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Resizing VHD C:\openstack\instances\instance-00000016\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:51:18.438 4076 3051055217480 GreenThread-11 DEBUG nova.policy [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'beffe9ba8529426fbb27e549254bd679', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91237ebda1b54d2dbc761e48fdd93516', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:51:18.438 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] WMI job succeeded: {'InstanceID': '44573840-6F6D-473E-A69A-839D25E852F1', 'Description': 'Creating Remote Virtual Machine', 'ElementName': 'Creating Remote Virtual Machine', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=362567), 'Cancellable': True, 'JobType': 300, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:18.455 4076 3051093168200 MainThread INFO nova.compute.manager [req-548e588a-a11e-4b46-857d-1714717a9616 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:18.455 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-548e588a-a11e-4b46-857d-1714717a9616 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:18.455 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-548e588a-a11e-4b46-857d-1714717a9616 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:18.661 4076 3051055217128 MainThread DEBUG os_win.utils.compute.livemigrationutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Replacing host resource \\N-H1-808474-1\root\virtualization\v2:Msvm_DiskDrive.CreationClassName="Msvm_DiskDrive",DeviceID="Microsoft:353B3BE8-310C-4cf4-839E-4E1B14616136\\1",SystemCreationClassName="Msvm_ComputerSystem",SystemName="N-H1-808474-1" with \\N-H1-808474-1\root\virtualization\v2:Msvm_DiskDrive.CreationClassName="Msvm_DiskDrive",DeviceID="Microsoft:353B3BE8-310C-4cf4-839E-4E1B14616136\\1",SystemCreationClassName="Msvm_ComputerSystem",SystemName="N-H1-808474-1" on planned VM instance-0000000e _update_planned_vm_disk_resources c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:128 2021-09-10 16:51:18.661 4076 3051055217128 MainThread DEBUG os_win.utils.compute.livemigrationutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Updating remote planned VM disk paths for VM: instance-0000000e _update_planned_vm_disk_resources c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:133 2021-09-10 16:51:18.692 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] driver pre_live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={4e982f7d-54f9-44a5-9572-7ad727331d99='f3cb7a9b-49be-4e16-8af7-6ddc6b51fc20'},vifs=[VIFMigrateData],wait_for_vif_plugged=) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8230 2021-09-10 16:51:19.267 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:19.350 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:51:19.505 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:19.518 4076 3051093169608 MainThread DEBUG nova.virt.hardware [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:51:19.518 4076 3051093169608 MainThread INFO nova.compute.claims [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Claim successful on node n-h1-808474-1 2021-09-10 16:51:19.751 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:19.787 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:19.787 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Port 9ce6624e-4307-41bc-9d39-c5c0bd6572b9 updated with migration profile {'migrating_to': 'n-h1-808474-1'} successfully _setup_migration_port_profile c:\openstack\build\nova\nova\network\neutron.py:343 2021-09-10 16:51:19.907 4076 3051093169256 MainThread INFO nova.compute.manager [req-e99836d1-ff36-435b-84f1-27256ad3d8b7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:19.907 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [req-e99836d1-ff36-435b-84f1-27256ad3d8b7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:19.907 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [req-e99836d1-ff36-435b-84f1-27256ad3d8b7 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:19.959 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] pre_live_migration result data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids={4e982f7d-54f9-44a5-9572-7ad727331d99='f3cb7a9b-49be-4e16-8af7-6ddc6b51fc20'},vifs=[VIFMigrateData],wait_for_vif_plugged=True) pre_live_migration c:\openstack\build\nova\nova\compute\manager.py:8285 2021-09-10 16:51:20.011 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:20.079 4076 3051093169608 MainThread DEBUG nova.compute.provider_tree [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:20.097 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:20.111 4076 3051093169608 MainThread DEBUG nova.scheduler.client.report [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:20.111 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:20.111 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:51:20.136 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:51:20.266 4076 3051093169608 MainThread DEBUG nova.compute.utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:20.266 4076 3051055217128 GreenThread-12 DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:51:20.266 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:51:20.301 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Successfully created port: f84ec91f-cb9d-48b1-a100-212291633d95 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:51:20.301 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:51:20.301 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:20.322 4076 3051055214664 MainThread DEBUG nova.virt.hardware [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:51:20.322 4076 3051055214664 MainThread INFO nova.compute.claims [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Claim successful on node n-h1-808474-1 2021-09-10 16:51:20.537 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:20.552 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:20.596 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:51:20.615 4076 3051055217128 GreenThread-12 DEBUG nova.policy [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '68a189ea096f4b08ae59e252907de4ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be36af928b5149e58720b1f68dda425d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:51:20.615 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Spawning new instance 2021-09-10 16:51:20.646 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:20.646 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:20.646 4076 3051093169608 MainThread DEBUG nova.block_device [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:20.646 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:20.646 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:20.659 4076 3051093169608 MainThread DEBUG os_win.utils.pathutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating directory: C:\openstack\instances\instance-00000017 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:20.659 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000017\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:51:20.704 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Resizing VHD C:\openstack\instances\instance-00000017\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:51:20.817 4076 3051055214664 MainThread DEBUG nova.compute.provider_tree [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:20.848 4076 3051055214664 MainThread DEBUG nova.scheduler.client.report [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:20.848 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:20.848 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:51:21.007 4076 3051055214664 MainThread DEBUG nova.compute.utils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:21.007 4076 3051093168552 GreenThread-13 DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:51:21.007 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:51:21.054 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:51:21.301 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:51:21.316 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Spawning new instance 2021-09-10 16:51:21.316 4076 3051093169784 MainThread INFO nova.compute.manager [req-42d2b43f-407f-4ac0-af54-8a007f3158e5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:21.316 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-42d2b43f-407f-4ac0-af54-8a007f3158e5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:21.330 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-42d2b43f-407f-4ac0-af54-8a007f3158e5 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:21.330 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000018" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:21.330 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000018" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:21.330 4076 3051055214664 MainThread DEBUG nova.block_device [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:21.330 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:21.330 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:21.344 4076 3051093168552 GreenThread-13 DEBUG nova.policy [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '68a189ea096f4b08ae59e252907de4ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be36af928b5149e58720b1f68dda425d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:51:21.344 4076 3051055214664 MainThread DEBUG os_win.utils.pathutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating directory: C:\openstack\instances\instance-00000018 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:21.344 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000018\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:51:21.416 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Resizing VHD C:\openstack\instances\instance-00000018\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:51:21.815 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:21.815 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 16:51:21.845 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 16:51:21.845 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:21.845 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 16:51:21.860 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:22.439 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Successfully updated port: f84ec91f-cb9d-48b1-a100-212291633d95 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:22.485 4076 3051055217480 GreenThread-11 DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Acquired lock "refresh_cache-59feb82d-9fd9-4170-aa08-742104f9e642" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:22.485 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:22.689 4076 3051093169960 MainThread INFO nova.compute.manager [req-40be9e89-879e-4c6c-aa43-cbe50f85c195 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:22.689 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-40be9e89-879e-4c6c-aa43-cbe50f85c195 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:22.689 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-40be9e89-879e-4c6c-aa43-cbe50f85c195 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:22.712 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:22.735 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Successfully created port: 07cb4db7-9b5e-4c96-ba70-2abd8300eb8a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:51:22.801 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Received event network-changed-f84ec91f-cb9d-48b1-a100-212291633d95 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:22.801 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Refreshing instance network info cache due to event network-changed-f84ec91f-cb9d-48b1-a100-212291633d95. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:23.067 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Successfully created port: 0110d67d-01ad-4b98-9caf-80b01bc70bfb _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:51:23.722 4076 3051055217480 GreenThread-11 DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Updating instance_info_cache with network_info: [{"id": "f84ec91f-cb9d-48b1-a100-212291633d95", "address": "fa:16:3e:61:bc:31", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf84ec91f-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:23.753 4076 3051055217480 GreenThread-11 DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Releasing lock "refresh_cache-59feb82d-9fd9-4170-aa08-742104f9e642" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:23.753 4076 3051055217480 GreenThread-11 DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Instance network_info: |[{"id": "f84ec91f-cb9d-48b1-a100-212291633d95", "address": "fa:16:3e:61:bc:31", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf84ec91f-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:23.753 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-59feb82d-9fd9-4170-aa08-742104f9e642" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:23.753 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Refreshing network info cache for port f84ec91f-cb9d-48b1-a100-212291633d95 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:23.753 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Preparing to wait for external event network-vif-plugged-f84ec91f-cb9d-48b1-a100-212291633d95 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:23.773 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:23.773 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:23.773 4076 3051055217304 MainThread DEBUG os_win.utils.compute.vmutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating VM instance-00000016 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:23.873 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:23.873 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:51:24.224 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:24.224 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:51:24.469 4076 3051093170312 MainThread INFO nova.compute.manager [req-d7816ec1-19d4-4cc4-8ad9-d3a4a2abdb37 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:24.469 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-d7816ec1-19d4-4cc4-8ad9-d3a4a2abdb37 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:24.469 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-d7816ec1-19d4-4cc4-8ad9-d3a4a2abdb37 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:24.551 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Successfully updated port: 07cb4db7-9b5e-4c96-ba70-2abd8300eb8a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:24.580 4076 3051055217128 GreenThread-12 DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Acquired lock "refresh_cache-9446e666-4364-4dcd-b041-405653a856bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:24.580 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:24.663 4076 3051093168200 MainThread DEBUG nova.compute.manager [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Received event network-changed-07cb4db7-9b5e-4c96-ba70-2abd8300eb8a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:24.675 4076 3051093168200 MainThread DEBUG nova.compute.manager [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Refreshing instance network info cache due to event network-changed-07cb4db7-9b5e-4c96-ba70-2abd8300eb8a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:24.675 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Successfully updated port: 0110d67d-01ad-4b98-9caf-80b01bc70bfb _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:24.717 4076 3051093168552 GreenThread-13 DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:24.717 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:24.741 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:24.758 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:24.963 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:25.596 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Updated VIF entry in instance network info cache for port f84ec91f-cb9d-48b1-a100-212291633d95. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:25.596 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Updating instance_info_cache with network_info: [{"id": "f84ec91f-cb9d-48b1-a100-212291633d95", "address": "fa:16:3e:61:bc:31", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf84ec91f-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:25.613 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-307e61bf-b67c-4385-a6f6-25696925f24d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-59feb82d-9fd9-4170-aa08-742104f9e642" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:25.849 4076 3051055217128 GreenThread-12 DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Updating instance_info_cache with network_info: [{"id": "07cb4db7-9b5e-4c96-ba70-2abd8300eb8a", "address": "fa:16:3e:5f:89:0c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.93", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap07cb4db7-9b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:25.895 4076 3051055217128 GreenThread-12 DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Releasing lock "refresh_cache-9446e666-4364-4dcd-b041-405653a856bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:25.895 4076 3051055217128 GreenThread-12 DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Instance network_info: |[{"id": "07cb4db7-9b5e-4c96-ba70-2abd8300eb8a", "address": "fa:16:3e:5f:89:0c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.93", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap07cb4db7-9b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:25.895 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Preparing to wait for external event network-vif-plugged-07cb4db7-9b5e-4c96-ba70-2abd8300eb8a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:25.895 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:25.895 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:25.895 4076 3051093169608 MainThread DEBUG os_win.utils.compute.vmutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating VM instance-00000017 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:25.895 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9446e666-4364-4dcd-b041-405653a856bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:25.895 4076 3051093168200 MainThread DEBUG nova.network.neutron [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Refreshing network info cache for port 07cb4db7-9b5e-4c96-ba70-2abd8300eb8a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:25.971 4076 3051093168552 GreenThread-13 DEBUG nova.network.neutron [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:26.004 4076 3051093168552 GreenThread-13 DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:26.004 4076 3051093168552 GreenThread-13 DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Instance network_info: |[{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:26.004 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Preparing to wait for external event network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:26.004 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:26.004 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:26.004 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating VM instance-00000018 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:26.159 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:26.176 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:26.176 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:51:26.176 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:26.205 4076 3051055217480 MainThread INFO nova.compute.manager [req-804218de-7db1-4818-9bcf-0fc2a372ea13 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:26.205 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-804218de-7db1-4818-9bcf-0fc2a372ea13 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:26.208 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-804218de-7db1-4818-9bcf-0fc2a372ea13 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:26.208 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:26.208 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:26.242 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:26.242 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:26.266 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:26.266 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:26.266 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:51:26.266 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:26.298 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:26.613 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=4705MB free_disk=54GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:51:26.613 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:26.754 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received event network-changed-0110d67d-01ad-4b98-9caf-80b01bc70bfb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:26.754 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Refreshing instance network info cache due to event network-changed-0110d67d-01ad-4b98-9caf-80b01bc70bfb. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:26.754 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:26.754 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Refreshing network info cache for port 0110d67d-01ad-4b98-9caf-80b01bc70bfb _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:26.798 4076 3051093168200 MainThread DEBUG nova.network.neutron [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Updated VIF entry in instance network info cache for port 07cb4db7-9b5e-4c96-ba70-2abd8300eb8a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:26.798 4076 3051093168200 MainThread DEBUG nova.network.neutron [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Updating instance_info_cache with network_info: [{"id": "07cb4db7-9b5e-4c96-ba70-2abd8300eb8a", "address": "fa:16:3e:5f:89:0c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.93", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap07cb4db7-9b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:26.838 4076 3051093168200 MainThread DEBUG oslo_concurrency.lockutils [req-51c1122b-f0e1-44b1-88be-7d8741932321 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9446e666-4364-4dcd-b041-405653a856bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:26.847 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Migration for instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 refers to another host's instance! _pair_instances_to_migrations c:\openstack\build\nova\nova\compute\resource_tracker.py:911 2021-09-10 16:51:26.904 4076 3051049642296 GreenThread-2 INFO nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating resource usage from migration 6caf2594-4ef9-4a93-9694-0c999f81f712 2021-09-10 16:51:26.904 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Starting to track incoming migration 6caf2594-4ef9-4a93-9694-0c999f81f712 with flavor 452 _update_usage_from_migration c:\openstack\build\nova\nova\compute\resource_tracker.py:1330 2021-09-10 16:51:27.021 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:27.141 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:27.313 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.313 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.313 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 59feb82d-9fd9-4170-aa08-742104f9e642 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.313 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.313 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.331 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9446e666-4364-4dcd-b041-405653a856bd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.331 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 WARNING nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 has been moved to another host n-h2-808474-1(n-h2-808474-1). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:51:27.354 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:51:27.441 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 16:51:27.505 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 16:51:27.505 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 16:51:27.533 4076 3051093168904 MainThread INFO nova.compute.manager [req-fa844570-461b-4124-9ccf-40ce1d335fa3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:27.533 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-fa844570-461b-4124-9ccf-40ce1d335fa3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:27.533 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-fa844570-461b-4124-9ccf-40ce1d335fa3 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:27.533 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 16:51:27.628 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 16:51:27.647 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:27.657 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:27.801 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updated VIF entry in instance network info cache for port 0110d67d-01ad-4b98-9caf-80b01bc70bfb. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:27.801 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:27.822 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-12677804-09d6-4856-a8ba-4001b481945d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:27.922 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:27.943 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:27.943 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:51:27.943 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 1.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:28.503 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:28.503 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:28.532 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:51:28.532 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:51:28.650 4076 3051055217480 MainThread INFO nova.compute.manager [req-b16c825b-d97e-45ee-9a2c-5350ba464724 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:28.650 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-b16c825b-d97e-45ee-9a2c-5350ba464724 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:28.650 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-b16c825b-d97e-45ee-9a2c-5350ba464724 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:29.719 4076 3051055217480 MainThread INFO nova.compute.manager [req-ad627ab2-a17c-4c27-864f-fc0c0096006f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:29.719 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-ad627ab2-a17c-4c27-864f-fc0c0096006f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:29.719 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-ad627ab2-a17c-4c27-864f-fc0c0096006f 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:31.304 4076 3051055217480 MainThread INFO nova.compute.manager [req-86a1e7a5-695a-4347-833d-aff4f5cd1d23 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:31.304 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-86a1e7a5-695a-4347-833d-aff4f5cd1d23 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:31.304 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-86a1e7a5-695a-4347-833d-aff4f5cd1d23 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:31.673 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-e3bd6148-c1a6-424f-bfe4-3f9803829923 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Received event network-vif-plugged-f84ec91f-cb9d-48b1-a100-212291633d95 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:31.673 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-e3bd6148-c1a6-424f-bfe4-3f9803829923 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:31.693 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-e3bd6148-c1a6-424f-bfe4-3f9803829923 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:31.693 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-e3bd6148-c1a6-424f-bfe4-3f9803829923 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Processing event network-vif-plugged-f84ec91f-cb9d-48b1-a100-212291633d95 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:31.735 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Using config drive for instance 2021-09-10 16:51:32.439 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Creating config drive at C:\openstack\instances\instance-00000016\configdrive.iso 2021-09-10 16:51:32.485 4076 3051055217304 MainThread DEBUG oslo_concurrency.processutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000016\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpr_xwz2eu execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:34.940 4076 3051093168904 MainThread INFO nova.compute.manager [req-07a8b60c-4675-4133-aa36-d518d86f38f2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Get console output 2021-09-10 16:51:34.940 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-07a8b60c-4675-4133-aa36-d518d86f38f2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:34.973 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-07a8b60c-4675-4133-aa36-d518d86f38f2 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-00000006" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received event network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Processing event network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Received event network-vif-plugged-07cb4db7-9b5e-4c96-ba70-2abd8300eb8a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG oslo_concurrency.lockutils [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:35.131 4076 3051093168904 MainThread DEBUG nova.compute.manager [req-ef859f87-ac3c-4e47-9675-26e88b013c30 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Processing event network-vif-plugged-07cb4db7-9b5e-4c96-ba70-2abd8300eb8a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:35.178 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Using config drive for instance 2021-09-10 16:51:35.189 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Using config drive for instance 2021-09-10 16:51:35.563 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Creating config drive at C:\openstack\instances\instance-00000018\configdrive.iso 2021-09-10 16:51:35.627 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000018\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpav0e6ngt execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:37.739 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Creating config drive at C:\openstack\instances\instance-00000017\configdrive.iso 2021-09-10 16:51:37.782 4076 3051093169608 MainThread DEBUG oslo_concurrency.processutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000017\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp0ecb43lo execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:37.957 4076 3051055217304 MainThread DEBUG oslo_concurrency.processutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000016\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpr_xwz2eu" returned: 0 in 5.469s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:37.988 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000018\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpav0e6ngt" returned: 0 in 2.359s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:38.001 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:38.001 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:38.018 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:38.018 4076 3051093170488 MainThread ERROR nova.compute.manager [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Traceback (most recent call last): 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._delete_instance(context, instance, bdms) 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] NameError: name 'instant' is not defined 2021-09-10 16:51:38.018 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] 2021-09-10 16:51:38.033 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:38.033 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] VM Started (Lifecycle Event) 2021-09-10 16:51:38.047 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:38.064 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-6aa1fdcd-3be2-47be-846d-f6dc80fbb0e3 - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:38.064 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-6aa1fdcd-3be2-47be-846d-f6dc80fbb0e3 - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:38.127 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000000e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.079s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:38.127 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:38.174 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-6aa1fdcd-3be2-47be-846d-f6dc80fbb0e3 - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Synchronizing instance power state after lifecycle event "Started"; current vm_state: active, current task_state: migrating, current DB power_state: 1, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:38.206 4076 3051093169608 MainThread DEBUG oslo_concurrency.processutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000017\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp0ecb43lo" returned: 0 in 0.422s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:38.222 4076 3051055217480 MainThread INFO nova.compute.manager [req-6aa1fdcd-3be2-47be-846d-f6dc80fbb0e3 - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] During the sync_power process the instance has moved from host n-h2-808474-1 to host n-h1-808474-1 2021-09-10 16:51:38.285 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:38.332 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:38.360 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:38.549 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Setting boot order for instance: instance-00000016: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:38.613 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Setting boot order for instance: instance-00000018: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:38.692 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:38.692 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:38.692 4076 3051093170488 MainThread DEBUG nova.compute.provider_tree [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:38.722 4076 3051093170488 MainThread DEBUG nova.scheduler.client.report [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:38.722 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:38.722 4076 3051093170488 MainThread INFO nova.compute.manager [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:51:38.742 4076 3051093170488 MainThread ERROR oslo_messaging.rpc.server [req-b39cd12c-7091-4cb7-a179-743557e1ea54 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:51:38.742 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:51:38.767 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Setting boot order for instance: instance-00000017: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:38.767 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '96833286-7104-4698-84C8-D7D25401E5C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=40138), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:38.784 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '26EE042B-3F28-4C57-8063-A48FF49C8889', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=48600), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:38.818 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:38.861 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': 'CEE3CAB1-3DE8-44EA-80C3-3D48A65DBC38', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=26088), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:39.085 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:39.118 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000018" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:39.118 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:39.118 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] VM Started (Lifecycle Event) 2021-09-10 16:51:39.118 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:51:39.141 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-8d6b773e-b65b-4ac4-b652-ace4049f0495 - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:39.141 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-8d6b773e-b65b-4ac4-b652-ace4049f0495 - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:39.173 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job succeeded: {'InstanceID': '26EE042B-3F28-4C57-8063-A48FF49C8889', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=346271), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:39.173 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Successfully changed vm state of instance-00000018 to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:51:39.173 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Successfully changed state of VM instance-00000018 to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:51:39.173 4076 3051055214664 MainThread INFO nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Took 17.88 seconds to spawn the instance on the hypervisor. 2021-09-10 16:51:39.173 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:39.173 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:39.173 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000018" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.062s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:39.251 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-8d6b773e-b65b-4ac4-b652-ace4049f0495 - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:39.301 4076 3051055217480 MainThread INFO nova.compute.manager [req-8d6b773e-b65b-4ac4-b652-ace4049f0495 - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:39.394 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:39.394 4076 3051055217128 MainThread DEBUG nova.virt.hardware [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:51:39.394 4076 3051055217128 MainThread INFO nova.compute.claims [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Claim successful on node n-h1-808474-1 2021-09-10 16:51:39.439 4076 3051055214664 MainThread INFO nova.compute.manager [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Took 19.19 seconds to build instance. 2021-09-10 16:51:39.472 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-8467074e-3e2a-4384-bd8a-4b4f9f73b301 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 19.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:39.643 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:39.672 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:39.722 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:39.753 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:51:39.988 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:40.002 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:40.017 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:40.017 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:51:40.017 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.094s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:40.017 4076 3051055217480 MainThread DEBUG nova.virt.hardware [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:51:40.032 4076 3051055217480 MainThread INFO nova.compute.claims [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Claim successful on node n-h1-808474-1 2021-09-10 16:51:40.142 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:40.142 4076 3051055214664 GreenThread-14 DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:51:40.142 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:51:40.188 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:51:40.252 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:40.267 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:40.394 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:51:40.411 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Spawning new instance 2021-09-10 16:51:40.411 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:40.411 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:40.411 4076 3051055217128 MainThread DEBUG nova.block_device [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:40.428 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:40.428 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:40.428 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating directory: C:\openstack\instances\instance-0000001a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:40.439 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000001a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:51:40.439 4076 3051055214664 GreenThread-14 DEBUG nova.policy [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bf4dce0601aa49c0b9ea6c48f2953557', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ae864ac350142778dcbcb12a8b3800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:51:40.493 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Resizing VHD C:\openstack\instances\instance-0000001a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:51:40.595 4076 3051055217480 MainThread DEBUG nova.compute.provider_tree [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:40.614 4076 3051055217480 MainThread DEBUG nova.scheduler.client.report [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:40.614 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:40.614 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:51:40.753 4076 3051055217480 MainThread DEBUG nova.compute.utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:40.767 4076 3051093168552 GreenThread-15 DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:51:40.767 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:51:40.791 4076 3051055217480 MainThread DEBUG nova.compute.utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:51:40.816 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:51:41.048 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:51:41.063 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Spawning new instance 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG nova.block_device [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] block_device_list ['sdb'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG os_win.utils.pathutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Creating directory: C:\openstack\instances\instance-0000001b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:51:41.080 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000001b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:51:41.157 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Resizing VHD C:\openstack\instances\instance-0000001b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:51:41.283 4076 3051093168552 GreenThread-15 DEBUG nova.policy [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66161d18b31e45309aee94f739bc6a3f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '145f8177efc74f5e9be31c5f74e69d0a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:51:41.751 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Successfully created port: 4945a27a-6bd7-45af-aafa-b02bc482a981 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:51:43.065 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Successfully updated port: 4945a27a-6bd7-45af-aafa-b02bc482a981 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:43.095 4076 3051055214664 GreenThread-14 DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Acquired lock "refresh_cache-7b215f4d-04c3-4607-aa80-c5dbc91487ee" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:43.095 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:43.205 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:44.208 4076 3051055214664 GreenThread-14 DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Updating instance_info_cache with network_info: [{"id": "4945a27a-6bd7-45af-aafa-b02bc482a981", "address": "fa:16:3e:c1:f5:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.33", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4945a27a-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:44.243 4076 3051055214664 GreenThread-14 DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Releasing lock "refresh_cache-7b215f4d-04c3-4607-aa80-c5dbc91487ee" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:44.243 4076 3051055214664 GreenThread-14 DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Instance network_info: |[{"id": "4945a27a-6bd7-45af-aafa-b02bc482a981", "address": "fa:16:3e:c1:f5:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.33", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4945a27a-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:44.243 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Preparing to wait for external event network-vif-plugged-4945a27a-6bd7-45af-aafa-b02bc482a981 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:44.243 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:44.253 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:44.253 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating VM instance-0000001a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:44.328 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Successfully created port: 7da5b8c5-4a5a-4395-9386-dd7cf483c679 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:51:45.078 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Received event network-changed-4945a27a-6bd7-45af-aafa-b02bc482a981 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:45.096 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Refreshing instance network info cache due to event network-changed-4945a27a-6bd7-45af-aafa-b02bc482a981. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:45.096 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-7b215f4d-04c3-4607-aa80-c5dbc91487ee" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:45.096 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Refreshing network info cache for port 4945a27a-6bd7-45af-aafa-b02bc482a981 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:45.349 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:45.816 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Updated VIF entry in instance network info cache for port 4945a27a-6bd7-45af-aafa-b02bc482a981. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:45.816 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Updating instance_info_cache with network_info: [{"id": "4945a27a-6bd7-45af-aafa-b02bc482a981", "address": "fa:16:3e:c1:f5:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.33", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4945a27a-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:45.828 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-b724c8de-8329-43ab-aa3c-11dc5371f825 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-7b215f4d-04c3-4607-aa80-c5dbc91487ee" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:46.595 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Successfully updated port: 7da5b8c5-4a5a-4395-9386-dd7cf483c679 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:51:46.627 4076 3051093168552 GreenThread-15 DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Acquired lock "refresh_cache-f3ef8889-0aba-4544-aa33-a09b43d286aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:46.627 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:46.719 4076 3051093169256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:46.719 4076 3051093169256 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:46.719 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:46.719 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:46.735 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CEE3CAB1-3DE8-44EA-80C3-3D48A65DBC38', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=465619), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:46.735 4076 3051093169608 MainThread DEBUG os_win._utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 7.916965007781982 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:46.735 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '96833286-7104-4698-84C8-D7D25401E5C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=385883), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:46.735 4076 3051055217304 MainThread DEBUG os_win._utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 8.042851448059082 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:46.735 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-1630161a-86f3-457c-8595-9511bfff061e - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:46.735 4076 3051093169256 MainThread DEBUG nova.virt.hyperv.vmops [req-1630161a-86f3-457c-8595-9511bfff061e - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:46.752 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:51:46.814 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-1630161a-86f3-457c-8595-9511bfff061e - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:46.831 4076 3051093169256 MainThread INFO nova.compute.manager [req-1630161a-86f3-457c-8595-9511bfff061e - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:47.704 4076 3051093168552 GreenThread-15 DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Updating instance_info_cache with network_info: [{"id": "7da5b8c5-4a5a-4395-9386-dd7cf483c679", "address": "fa:16:3e:ce:c6:39", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7da5b8c5-4a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:47.720 4076 3051093168552 GreenThread-15 DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Releasing lock "refresh_cache-f3ef8889-0aba-4544-aa33-a09b43d286aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:47.720 4076 3051093168552 GreenThread-15 DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Instance network_info: |[{"id": "7da5b8c5-4a5a-4395-9386-dd7cf483c679", "address": "fa:16:3e:ce:c6:39", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7da5b8c5-4a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:51:47.720 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Preparing to wait for external event network-vif-plugged-7da5b8c5-4a5a-4395-9386-dd7cf483c679 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:51:47.720 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:47.720 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:47.720 4076 3051055217480 MainThread DEBUG os_win.utils.compute.vmutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Creating VM instance-0000001b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:51:47.769 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '059273CD-2539-450C-A348-34B52C07ADBF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20386), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:47.769 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'A8EADDEA-0350-4449-BF43-E7776FE19026', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21989), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:47.958 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '059273CD-2539-450C-A348-34B52C07ADBF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=134134), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:47.958 4076 3051093169608 MainThread DEBUG os_win._utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 9.139858961105347 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:47.958 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:47.958 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 9446e666-4364-4dcd-b041-405653a856bd] VM Stopped (Lifecycle Event) 2021-09-10 16:51:47.973 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:47.973 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:47.984 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-1b54bb47-0992-464b-857a-632348e227ec - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:47.984 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-1b54bb47-0992-464b-857a-632348e227ec - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:48.064 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-1b54bb47-0992-464b-857a-632348e227ec - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:48.097 4076 3051055214664 MainThread INFO nova.compute.manager [req-1b54bb47-0992-464b-857a-632348e227ec - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:48.097 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:48.097 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:48.097 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:48.097 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:48.129 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-54ce1255-3449-48b2-a07e-cedeb4883dc8 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:48.129 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-54ce1255-3449-48b2-a07e-cedeb4883dc8 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:48.188 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A8EADDEA-0350-4449-BF43-E7776FE19026', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=280093), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:48.188 4076 3051055217304 MainThread DEBUG os_win._utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 9.496007442474365 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:48.205 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-54ce1255-3449-48b2-a07e-cedeb4883dc8 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:48.220 4076 3051055214664 MainThread INFO nova.compute.manager [req-54ce1255-3449-48b2-a07e-cedeb4883dc8 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:49.003 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '8FACD378-3F88-4BEE-9175-458983847C75', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23133), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:49.181 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:49.181 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 9446e666-4364-4dcd-b041-405653a856bd] VM Stopped (Lifecycle Event) 2021-09-10 16:51:49.191 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:49.191 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:49.191 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8FACD378-3F88-4BEE-9175-458983847C75', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=131978), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:49.191 4076 3051093169608 MainThread DEBUG os_win._utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 10.372520208358765 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:49.208 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-2ca965a0-0c0d-4ad2-9726-f3137e119c41 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:49.208 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-2ca965a0-0c0d-4ad2-9726-f3137e119c41 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:49.255 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'F6594311-73C9-4D80-9EC1-0652F1A291E8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=44535), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:49.303 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-2ca965a0-0c0d-4ad2-9726-f3137e119c41 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:49.346 4076 3051055214664 MainThread INFO nova.compute.manager [req-2ca965a0-0c0d-4ad2-9726-f3137e119c41 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:49.441 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:49.441 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:49.441 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:49.463 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:49.463 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F6594311-73C9-4D80-9EC1-0652F1A291E8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=170493), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:49.463 4076 3051055217304 MainThread DEBUG os_win._utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 10.770606756210327 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:49.477 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-df371089-6a88-4e36-bc07-a91c5dbd37e4 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:49.477 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-df371089-6a88-4e36-bc07-a91c5dbd37e4 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:49.521 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:51:49.539 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-df371089-6a88-4e36-bc07-a91c5dbd37e4 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:49.564 4076 3051055214664 MainThread INFO nova.compute.manager [req-df371089-6a88-4e36-bc07-a91c5dbd37e4 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:50.238 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '6BDB1C90-EBEC-4382-9FEB-8DF45EA2998A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17584), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:50.425 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:50.425 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:50.425 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:50.425 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 9446e666-4364-4dcd-b041-405653a856bd] VM Stopped (Lifecycle Event) 2021-09-10 16:51:50.444 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-87591bb5-8ca9-4853-a03d-eb4bb5fe9e27 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:50.444 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-87591bb5-8ca9-4853-a03d-eb4bb5fe9e27 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:50.500 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6BDB1C90-EBEC-4382-9FEB-8DF45EA2998A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=157105), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:50.500 4076 3051093169608 MainThread DEBUG os_win._utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 11.681861639022827 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:50.521 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-87591bb5-8ca9-4853-a03d-eb4bb5fe9e27 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:50.536 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '1FAE6E0A-1CB8-431E-A4A5-C7DE28A86B28', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23329), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:50.536 4076 3051055214664 MainThread INFO nova.compute.manager [req-87591bb5-8ca9-4853-a03d-eb4bb5fe9e27 - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:50.719 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:50.719 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:50.719 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:50.719 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:50.736 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ee81e9b0-f72e-455c-9555-6b2c81dbb214 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:50.736 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-ee81e9b0-f72e-455c-9555-6b2c81dbb214 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:50.799 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ee81e9b0-f72e-455c-9555-6b2c81dbb214 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:50.799 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1FAE6E0A-1CB8-431E-A4A5-C7DE28A86B28', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=160387), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:50.799 4076 3051055217304 MainThread DEBUG os_win._utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 12.106130838394165 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:50.822 4076 3051055214664 MainThread INFO nova.compute.manager [req-ee81e9b0-f72e-455c-9555-6b2c81dbb214 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:51.113 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Received event network-changed-7da5b8c5-4a5a-4395-9386-dd7cf483c679 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:51.113 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Refreshing instance network info cache due to event network-changed-7da5b8c5-4a5a-4395-9386-dd7cf483c679. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:51:51.113 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f3ef8889-0aba-4544-aa33-a09b43d286aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:51.113 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Refreshing network info cache for port 7da5b8c5-4a5a-4395-9386-dd7cf483c679 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:51:51.548 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '696434D7-DF8A-413C-87EB-7AF301034C12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17874), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:51.704 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '696434D7-DF8A-413C-87EB-7AF301034C12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=130290), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:51.732 4076 3051093169608 MainThread DEBUG os_win._utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 12.913600206375122 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:51.732 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:51.732 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 9446e666-4364-4dcd-b041-405653a856bd] VM Stopped (Lifecycle Event) 2021-09-10 16:51:51.732 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:51.737 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:51.755 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-19829259-8f5f-4110-922a-3057062ce72a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:51.755 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-19829259-8f5f-4110-922a-3057062ce72a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:51.802 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-19829259-8f5f-4110-922a-3057062ce72a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:51.828 4076 3051093168552 MainThread INFO nova.compute.manager [req-19829259-8f5f-4110-922a-3057062ce72a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:51.860 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'D144D2CD-A6E2-44FE-9E0B-8CB17E181FEB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=43190), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:51.908 4076 3051093168552 MainThread INFO nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Post operation of migration started 2021-09-10 16:51:52.018 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:52.018 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:52.018 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:52.018 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:52.037 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D144D2CD-A6E2-44FE-9E0B-8CB17E181FEB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=132591), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:52.037 4076 3051055217304 MainThread DEBUG os_win._utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 13.344168186187744 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:52.037 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-dfbd5bab-a241-4a66-89e1-3ae7cf3cb996 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:52.037 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-dfbd5bab-a241-4a66-89e1-3ae7cf3cb996 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:52.048 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:51:52.048 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:51:52.099 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-dfbd5bab-a241-4a66-89e1-3ae7cf3cb996 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:52.126 4076 3051093169784 MainThread INFO nova.compute.manager [req-dfbd5bab-a241-4a66-89e1-3ae7cf3cb996 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:52.333 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Updated VIF entry in instance network info cache for port 7da5b8c5-4a5a-4395-9386-dd7cf483c679. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:51:52.333 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Updating instance_info_cache with network_info: [{"id": "7da5b8c5-4a5a-4395-9386-dd7cf483c679", "address": "fa:16:3e:ce:c6:39", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7da5b8c5-4a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:52.345 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-5c83f8c7-3056-4ff4-b040-7ba0e154a5c4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f3ef8889-0aba-4544-aa33-a09b43d286aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:52.564 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:52.586 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-57d0f466-544c-46c2-a26f-0d90a3a7756e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Received event network-vif-plugged-4945a27a-6bd7-45af-aafa-b02bc482a981 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:52.586 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-57d0f466-544c-46c2-a26f-0d90a3a7756e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:52.586 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-57d0f466-544c-46c2-a26f-0d90a3a7756e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:52.597 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-57d0f466-544c-46c2-a26f-0d90a3a7756e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Processing event network-vif-plugged-4945a27a-6bd7-45af-aafa-b02bc482a981 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:52.597 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:51:52.611 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:52.611 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.allocate_pci_devices_for_instance" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:52.611 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] post_live_migration_at_destination called post_live_migration_at_destination c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:124 2021-09-10 16:51:52.611 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:52.611 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:52.631 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Using config drive for instance 2021-09-10 16:51:52.704 4076 3051093168552 MainThread DEBUG nova.objects.instance [req-361454bc-34db-4d92-afb2-0647bf016643 631464703d054b39ab2dbb2ef12349bf 65f9947f24f541579df4a582376707cc - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Trying to apply a migration context that does not seem to be set for this instance apply_migration_context c:\openstack\build\nova\nova\objects\instance.py:1031 2021-09-10 16:51:52.768 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Creating config drive at C:\openstack\instances\instance-0000001a\configdrive.iso 2021-09-10 16:51:52.806 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmptzzbtj8i execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:53.024 4076 3051093170488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:53.024 4076 3051093170488 MainThread INFO nova.compute.manager [-] [instance: 9446e666-4364-4dcd-b041-405653a856bd] VM Stopped (Lifecycle Event) 2021-09-10 16:51:53.024 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:53.041 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:53.056 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-f9129287-e999-43e2-b9a4-9dcd3d19375a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:53.056 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9129287-e999-43e2-b9a4-9dcd3d19375a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:53.066 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000017\' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000017\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)instance-00000017F332D9CF-0A03-4C0B-B329-39CE73E8623E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B17C3F75-514B-4145-9169-DC756F5868F3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=133919), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\nNot enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)\n\n'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E)", 'ErrorSummaryDescription': "'instance-00000017' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000017.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:53.066 4076 3051093169608 MainThread ERROR nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Failed to change vm state of instance-00000017 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. 2021-09-10 16:51:53.066 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Got request to destroy instance 2021-09-10 16:51:53.098 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '435089D5-C343-4FDB-A417-25470127FDA7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=33463), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:53.188 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-f9129287-e999-43e2-b9a4-9dcd3d19375a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:53.220 4076 3051093170488 MainThread INFO nova.compute.manager [req-f9129287-e999-43e2-b9a4-9dcd3d19375a - - - - -] [instance: 9446e666-4364-4dcd-b041-405653a856bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:53.267 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmptzzbtj8i" returned: 0 in 0.453s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:53.282 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000016\' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000016\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)instance-00000016A062E414-F02C-41BA-BDC4-415F3993E26F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '435089D5-C343-4FDB-A417-25470127FDA7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=156750), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\nNot enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)\n\n'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F)", 'ErrorSummaryDescription': "'instance-00000016' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000016.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:53.282 4076 3051055217304 MainThread ERROR nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Failed to change vm state of instance-00000016 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. 2021-09-10 16:51:53.282 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Got request to destroy instance 2021-09-10 16:51:53.282 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:53.282 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:53.310 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:53.310 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] VM Stopped (Lifecycle Event) 2021-09-10 16:51:53.332 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-30848770-0f6a-44c1-bef4-46b5566a08b3 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:53.332 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-30848770-0f6a-44c1-bef4-46b5566a08b3 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:53.413 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-30848770-0f6a-44c1-bef4-46b5566a08b3 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:53.426 4076 3051055214664 MainThread INFO nova.compute.manager [req-30848770-0f6a-44c1-bef4-46b5566a08b3 - - - - -] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:53.816 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Setting boot order for instance: instance-0000001a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:53.894 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:53.938 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'F753F204-61D4-4EE0-B6D8-339CC6372442', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=27861), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:54.172 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:54.193 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:54.193 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:54.193 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:51:54.208 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-bdf3324e-e03c-49d8-8055-2833a9dffa0e - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:54.208 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-bdf3324e-e03c-49d8-8055-2833a9dffa0e - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:54.316 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F753F204-61D4-4EE0-B6D8-339CC6372442', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=182952), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:54.316 4076 3051055217128 MainThread DEBUG os_win._utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.4216880798339844 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:54.345 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-bdf3324e-e03c-49d8-8055-2833a9dffa0e - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:54.376 4076 3051055214664 MainThread INFO nova.compute.manager [req-bdf3324e-e03c-49d8-8055-2833a9dffa0e - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:55.384 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'F3B0C600-E21B-4730-94B7-DA36C725BB88', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=45745), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:55.566 4076 3051093170488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:55.566 4076 3051093170488 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:51:55.566 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:55.566 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:55.566 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F3B0C600-E21B-4730-94B7-DA36C725BB88', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=169630), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:55.566 4076 3051055217128 MainThread DEBUG os_win._utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.6714181900024414 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:55.594 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-196c4e87-dde9-4fad-b6f8-37c926f44475 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:55.594 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-196c4e87-dde9-4fad-b6f8-37c926f44475 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:55.594 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager.detach_volume..do_detach_volume" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:55.617 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-3406a0a9-c230-4827-a7b6-16241783a0e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Received event network-vif-plugged-7da5b8c5-4a5a-4395-9386-dd7cf483c679 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:51:55.617 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-3406a0a9-c230-4827-a7b6-16241783a0e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:55.617 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-3406a0a9-c230-4827-a7b6-16241783a0e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:55.617 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-3406a0a9-c230-4827-a7b6-16241783a0e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Processing event network-vif-plugged-7da5b8c5-4a5a-4395-9386-dd7cf483c679 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:51:55.628 4076 3051055214664 MainThread INFO nova.compute.manager [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Detaching volume 4e982f7d-54f9-44a5-9572-7ad727331d99 2021-09-10 16:51:55.662 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Using config drive for instance 2021-09-10 16:51:55.705 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-196c4e87-dde9-4fad-b6f8-37c926f44475 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:55.705 4076 3051055214664 MainThread INFO nova.virt.block_device [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Attempting to driver detach volume 4e982f7d-54f9-44a5-9572-7ad727331d99 from mountpoint /dev/sdb 2021-09-10 16:51:55.725 4076 3051093170488 MainThread INFO nova.compute.manager [req-196c4e87-dde9-4fad-b6f8-37c926f44475 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:55.736 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.volumeops [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Detaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99', 'target_lun': 0, 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'auth_method': 'CHAP', 'auth_username': '64HsfuxbJfZXqwRTayVC', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'attaching', 'instance': '17abaf80-43c7-4c09-b4a2-c12636cd0cc3', 'attached_at': '2021-09-10T16:51:11.000000', 'detached_at': '', 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'serial': '4e982f7d-54f9-44a5-9572-7ad727331d99'} from instance-0000000e detach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:128 2021-09-10 16:51:55.767 4076 3051055214664 MainThread DEBUG os_brick.initiator.connector [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Factory for iscsi on None factory c:\python37\lib\site-packages\os_brick\initiator\connector.py:282 2021-09-10 16:51:55.767 4076 3051055214664 MainThread INFO os_brick.initiator.windows.iscsi [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] No iSCSI initiator was explicitly requested. The Microsoft iSCSI initiator will choose the initiator when establishing sessions. 2021-09-10 16:51:55.767 4076 3051055214664 MainThread DEBUG os_brick.initiator.windows.iscsi [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] ==> get_volume_paths: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99', 'target_lun': 0, 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'auth_method': 'CHAP', 'auth_username': '64HsfuxbJfZXqwRTayVC', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:55.784 4076 3051055214664 MainThread DEBUG os_brick.initiator.windows.iscsi [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] <== get_volume_paths: return (17ms) ['\\\\.\\PhysicalDrive1'] trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:55.784 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Creating config drive at C:\openstack\instances\instance-0000001b\configdrive.iso 2021-09-10 16:51:55.863 4076 3051055217480 MainThread DEBUG oslo_concurrency.processutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmppl9geltj execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:51:56.016 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.volumeops [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Detaching disk \\N-H1-808474-1\root\virtualization\v2:Msvm_DiskDrive.CreationClassName="Msvm_DiskDrive",DeviceID="Microsoft:353B3BE8-310C-4cf4-839E-4E1B14616136\\1",SystemCreationClassName="Msvm_ComputerSystem",SystemName="N-H1-808474-1" from instance: instance-0000000e detach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:292 2021-09-10 16:51:56.347 4076 3051055214664 MainThread DEBUG os_brick.initiator.windows.iscsi [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99', 'target_lun': 0, 'volume_id': '4e982f7d-54f9-44a5-9572-7ad727331d99', 'auth_method': 'CHAP', 'auth_username': '64HsfuxbJfZXqwRTayVC', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:51:56.347 4076 3051055214664 MainThread DEBUG os_win.utils.storage.diskutils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:51:56.362 4076 3051055217480 MainThread DEBUG oslo_concurrency.processutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmppl9geltj" returned: 0 in 0.500s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:51:56.519 4076 3051055214664 MainThread DEBUG os_win.utils.storage.diskutils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:51:56.519 4076 3051055214664 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 16:51:56.542 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:51:56.549 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.549 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.563 4076 3051093169608 MainThread DEBUG os_win.utils.compute.vmutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Successfully changed vm state of instance-00000017 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:51:56.563 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Successfully changed state of VM instance-00000017 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:51:56.611 4076 3051055214664 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-4e982f7d-54f9-44a5-9572-7ad727331d99 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 16:51:56.611 4076 3051055214664 MainThread DEBUG os_brick.initiator.windows.iscsi [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] <== disconnect_volume: return (263ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:51:56.632 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '3C895149-328D-453E-90C0-E8CC6B581D1F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21233), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:56.669 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '49541755-2794-4877-8F5D-18487E2CE22F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=40918), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:56.723 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:51:56.723 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.723 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.751 4076 3051055217304 MainThread DEBUG os_win.utils.compute.vmutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed vm state of instance-00000016 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:51:56.751 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed state of VM instance-00000016 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:51:56.799 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3C895149-328D-453E-90C0-E8CC6B581D1F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=162568), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:56.799 4076 3051055217128 MainThread DEBUG os_win._utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.9048972129821777 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:56.819 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job succeeded: {'InstanceID': '49541755-2794-4877-8F5D-18487E2CE22F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=56195), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:56.819 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.819 4076 3051093169608 MainThread DEBUG os_win.utils.pathutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Removing directory: C:\openstack\instances\instance-00000017 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:51:56.819 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.819 4076 3051093169608 MainThread ERROR nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Traceback (most recent call last): 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(*args, **kwargs) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] admin_password, network_info, block_device_info) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(self, *args, **kwds) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] should_plug_vifs=False) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'req_state': req_state}) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] time.sleep(sleep_time) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return f(*args, **kwargs) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return self._wait_for_job(job_path) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] error_desc=err_desc) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] During handling of the above exception, another exception occurred: 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Traceback (most recent call last): 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] yield resources 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] accel_info=accel_info) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(*args, **kwargs) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] admin_password, network_info, block_device_info) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(self, *args, **kwds) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] should_plug_vifs=False) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'req_state': req_state}) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] time.sleep(sleep_time) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return f(*args, **kwargs) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return self._wait_for_job(job_path) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] error_desc=err_desc) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:56.819 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:56.829 4076 3051093169608 MainThread INFO nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Terminating instance 2021-09-10 16:51:56.829 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:51:56.829 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Got request to destroy instance 2021-09-10 16:51:56.829 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:56.829 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:51:56.829 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.829 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.829 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '64840B94-F6DB-4602-95CB-4F6A9109D875', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21393), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:56.860 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-d9fa6f20-c272-4831-aa31-0b9f1e2d6ddb - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:56.860 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-d9fa6f20-c272-4831-aa31-0b9f1e2d6ddb - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:56.860 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:51:56.860 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.860 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "instance-00000017" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.860 4076 3051093169608 MainThread INFO nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:51:56.860 4076 3051093169608 MainThread DEBUG nova.compute.claims [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:51:56.860 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.923 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Setting boot order for instance: instance-0000001b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:51:56.962 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job succeeded: {'InstanceID': '64840B94-F6DB-4602-95CB-4F6A9109D875', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=37563), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:51:56.962 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:56.962 4076 3051055217304 MainThread DEBUG os_win.utils.pathutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Removing directory: C:\openstack\instances\instance-00000016 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:51:56.962 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:56.962 4076 3051055217304 MainThread ERROR nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Traceback (most recent call last): 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(*args, **kwargs) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] admin_password, network_info, block_device_info) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(self, *args, **kwds) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] should_plug_vifs=False) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'req_state': req_state}) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] time.sleep(sleep_time) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return f(*args, **kwargs) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return self._wait_for_job(job_path) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] error_desc=err_desc) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During handling of the above exception, another exception occurred: 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Traceback (most recent call last): 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] yield resources 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] accel_info=accel_info) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(*args, **kwargs) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] admin_password, network_info, block_device_info) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(self, *args, **kwds) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] should_plug_vifs=False) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'req_state': req_state}) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] time.sleep(sleep_time) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return f(*args, **kwargs) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return self._wait_for_job(job_path) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] error_desc=err_desc) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:56.962 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:56.962 4076 3051055217304 MainThread INFO nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Terminating instance 2021-09-10 16:51:56.962 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:51:56.962 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Got request to destroy instance 2021-09-10 16:51:56.984 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-d9fa6f20-c272-4831-aa31-0b9f1e2d6ddb - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:56.987 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:57.003 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:51:57.003 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:57.003 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000016" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:57.003 4076 3051055217304 MainThread INFO nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 16:51:57.003 4076 3051055217304 MainThread DEBUG nova.compute.claims [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:51:57.003 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:51:57.018 4076 3051093168552 MainThread INFO nova.compute.manager [req-d9fa6f20-c272-4831-aa31-0b9f1e2d6ddb - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:57.018 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:57.053 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '6D78D9FF-3B43-4A34-8D06-604A408018CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21696), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:57.210 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6D78D9FF-3B43-4A34-8D06-604A408018CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=140017), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:57.219 4076 3051055217480 MainThread DEBUG os_win._utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.21598029136657715 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:57.219 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:57.219 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:51:57.238 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:57.238 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:57.253 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-8b3944d6-2512-4cd8-96b9-e9f10c449d4e - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:57.253 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-8b3944d6-2512-4cd8-96b9-e9f10c449d4e - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:57.300 4076 3051093169608 MainThread DEBUG nova.compute.provider_tree [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:57.317 4076 3051093169608 MainThread DEBUG nova.scheduler.client.report [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:57.317 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:57.317 4076 3051093169608 MainThread ERROR nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Traceback (most recent call last): 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(*args, **kwargs) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] admin_password, network_info, block_device_info) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(self, *args, **kwds) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] should_plug_vifs=False) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'req_state': req_state}) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] time.sleep(sleep_time) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return f(*args, **kwargs) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return self._wait_for_job(job_path) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] error_desc=err_desc) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] During handling of the above exception, another exception occurred: 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Traceback (most recent call last): 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] accel_info=accel_info) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(*args, **kwargs) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] admin_password, network_info, block_device_info) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return function(self, *args, **kwds) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] should_plug_vifs=False) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'req_state': req_state}) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] time.sleep(sleep_time) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self.force_reraise() 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] raise self.value 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return f(*args, **kwargs) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] return self._wait_for_job(job_path) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] error_desc=err_desc) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. 2021-09-10 16:51:57.317 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 9446e666-4364-4dcd-b041-405653a856bd] 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.compute.utils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Build of instance 9446e666-4364-4dcd-b041-405653a856bd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000017' failed to start. Not enough memory in the system to start the virtual machine instance-00000017. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000017' failed to start. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Not enough memory in the system to start the virtual machine instance-00000017 with ram size 1024 megabytes. (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) 'instance-00000017' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F332D9CF-0A03-4C0B-B329-39CE73E8623E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:51:57.331 4076 3051093169608 MainThread DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:51:57.347 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:57.347 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-8b3944d6-2512-4cd8-96b9-e9f10c449d4e - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:57.381 4076 3051093168552 MainThread INFO nova.compute.manager [req-8b3944d6-2512-4cd8-96b9-e9f10c449d4e - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:57.470 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:51:57.540 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:51:57.834 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:51:57.861 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:51:57.861 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.516s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:57.861 4076 3051055217304 MainThread ERROR nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Traceback (most recent call last): 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(*args, **kwargs) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] admin_password, network_info, block_device_info) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(self, *args, **kwds) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] should_plug_vifs=False) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'req_state': req_state}) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] time.sleep(sleep_time) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return f(*args, **kwargs) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return self._wait_for_job(job_path) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] error_desc=err_desc) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] During handling of the above exception, another exception occurred: 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Traceback (most recent call last): 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] accel_info=accel_info) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(*args, **kwargs) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] admin_password, network_info, block_device_info) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return function(self, *args, **kwds) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] should_plug_vifs=False) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'req_state': req_state}) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] time.sleep(sleep_time) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self.force_reraise() 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] raise self.value 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return f(*args, **kwargs) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] return self._wait_for_job(job_path) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] error_desc=err_desc) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. 2021-09-10 16:51:57.861 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] 2021-09-10 16:51:57.861 4076 3051055217304 MainThread DEBUG nova.compute.utils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:51:57.861 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Build of instance 59feb82d-9fd9-4170-aa08-742104f9e642 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000016' failed to start. Not enough memory in the system to start the virtual machine instance-00000016. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000016' failed to start. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Not enough memory in the system to start the virtual machine instance-00000016 with ram size 1024 megabytes. (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) 'instance-00000016' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A062E414-F02C-41BA-BDC4-415F3993E26F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:51:57.882 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:51:57.882 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:51:57.882 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:51:57.882 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:51:57.893 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '355EA23F-6C1B-4489-910D-4D0326A6C174', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22603), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:58.047 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:58.047 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:51:58.047 4076 3051095081032 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:58.047 4076 3051095081032 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:58.080 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '355EA23F-6C1B-4489-910D-4D0326A6C174', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=168094), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:58.080 4076 3051055217128 MainThread DEBUG os_win._utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.18567156791687 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:58.094 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-dd8276fa-4a9a-4561-8251-fd961be244e0 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:58.094 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-dd8276fa-4a9a-4561-8251-fd961be244e0 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:58.132 4076 3051093169608 MainThread DEBUG nova.network.neutron [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:58.166 4076 3051093169608 MainThread INFO nova.compute.manager [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 9446e666-4364-4dcd-b041-405653a856bd] Took 0.84 seconds to deallocate network for instance. 2021-09-10 16:51:58.172 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-dd8276fa-4a9a-4561-8251-fd961be244e0 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:58.194 4076 3051093169784 MainThread INFO nova.compute.manager [req-dd8276fa-4a9a-4561-8251-fd961be244e0 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:58.266 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '6A6D5FB1-1FE7-4469-9637-3A9E1B885A24', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22517), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:58.409 4076 3051093169608 MainThread INFO nova.scheduler.client.report [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Deleted allocations for instance 9446e666-4364-4dcd-b041-405653a856bd 2021-09-10 16:51:58.454 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6A6D5FB1-1FE7-4469-9637-3A9E1B885A24', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=132196), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:58.454 4076 3051055217480 MainThread DEBUG os_win._utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4503774642944336 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:58.474 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:58.474 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:58.474 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:58.474 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:51:58.485 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-6382f5fd-88e0-488a-a292-4e3a2359ec0f 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "9446e666-4364-4dcd-b041-405653a856bd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 39.219s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:58.505 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2205ec43-7741-4e10-81a1-4136afb53019 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:58.505 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-2205ec43-7741-4e10-81a1-4136afb53019 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:58.567 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2205ec43-7741-4e10-81a1-4136afb53019 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:58.584 4076 3051093169784 MainThread INFO nova.compute.manager [req-2205ec43-7741-4e10-81a1-4136afb53019 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:59.101 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:51:59.110 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'CAF66998-A225-41BE-8393-DD544E78DEE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20807), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:59.131 4076 3051055217304 MainThread INFO nova.compute.manager [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 59feb82d-9fd9-4170-aa08-742104f9e642] Took 1.25 seconds to deallocate network for instance. 2021-09-10 16:51:59.301 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:59.301 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:51:59.301 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:59.301 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:59.301 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CAF66998-A225-41BE-8393-DD544E78DEE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=157412), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:59.323 4076 3051055217128 MainThread DEBUG os_win._utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.428557634353638 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:59.331 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-3e10c748-4eff-45ad-ba16-ec2857420509 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:59.331 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-3e10c748-4eff-45ad-ba16-ec2857420509 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:59.393 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-3e10c748-4eff-45ad-ba16-ec2857420509 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:59.425 4076 3051055217304 MainThread INFO nova.scheduler.client.report [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Deleted allocations for instance 59feb82d-9fd9-4170-aa08-742104f9e642 2021-09-10 16:51:59.441 4076 3051093170312 MainThread INFO nova.compute.manager [req-3e10c748-4eff-45ad-ba16-ec2857420509 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:51:59.470 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-e4083828-d740-4043-bff5-e307aa22db1a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "59feb82d-9fd9-4170-aa08-742104f9e642" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 42.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:59.486 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '4A902EBC-7687-497B-80A2-7D5853E02C51', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21773), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:51:59.645 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:51:59.645 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:51:59.663 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:51:59.663 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:51:59.663 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4A902EBC-7687-497B-80A2-7D5853E02C51', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=130966), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 51, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 51, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:51:59.663 4076 3051055217480 MainThread DEBUG os_win._utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.659658432006836 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:51:59.672 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-c159bba4-bb5e-4810-bd4e-d0e5128258f9 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:51:59.672 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-c159bba4-bb5e-4810-bd4e-d0e5128258f9 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:51:59.751 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-c159bba4-bb5e-4810-bd4e-d0e5128258f9 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:51:59.784 4076 3051055217304 MainThread INFO nova.compute.manager [req-c159bba4-bb5e-4810-bd4e-d0e5128258f9 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:00.375 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'D8C9A137-2CA6-475D-AC1A-50BBCA018A11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=26774), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:00.552 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:00.552 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] VM Stopped (Lifecycle Event) 2021-09-10 16:52:00.552 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:00.552 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:00.564 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001a\' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC84604506010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)instance-0000001a6F9B5A07-5C7C-45E8-839D-AAC846045060%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D8C9A137-2CA6-475D-AC1A-50BBCA018A11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=158283), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\nNot enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)\n\n'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060)", 'ErrorSummaryDescription': "'instance-0000001a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:00.564 4076 3051055217128 MainThread ERROR nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Failed to change vm state of instance-0000001a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. 2021-09-10 16:52:00.564 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Got request to destroy instance 2021-09-10 16:52:00.579 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-08aa352d-38da-4aae-a7cb-6181075024c2 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:00.579 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-08aa352d-38da-4aae-a7cb-6181075024c2 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:00.657 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-08aa352d-38da-4aae-a7cb-6181075024c2 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:00.688 4076 3051055217304 MainThread INFO nova.compute.manager [req-08aa352d-38da-4aae-a7cb-6181075024c2 - - - - -] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:00.719 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': 'CA9C82EC-F31A-441D-9A46-C618D4C53216', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=24514), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:00.797 4076 3051055214664 MainThread DEBUG nova.objects.instance [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lazy-loading 'flavor' on Instance uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:52:00.862 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-a1a673b7-f5a6-481b-9740-d775ab28740b fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager.detach_volume..do_detach_volume" :: held 5.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:00.878 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CA9C82EC-F31A-441D-9A46-C618D4C53216', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=131772), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:00.878 4076 3051055217480 MainThread DEBUG os_win._utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8748269081115723 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:00.878 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:00.878 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:00.878 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:00.878 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:52:00.892 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-ea15bc38-574c-4145-95ff-98da17c2f916 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:00.912 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-ea15bc38-574c-4145-95ff-98da17c2f916 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:00.969 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-ea15bc38-574c-4145-95ff-98da17c2f916 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:00.991 4076 3051055217304 MainThread INFO nova.compute.manager [req-ea15bc38-574c-4145-95ff-98da17c2f916 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:01.939 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '778632A6-B0F0-4EF8-B76D-EFD3F43E605F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19386), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:02.109 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '778632A6-B0F0-4EF8-B76D-EFD3F43E605F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=143249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:02.119 4076 3051055217480 MainThread DEBUG os_win._utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.1160888671875 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:02.144 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:02.144 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:52:02.144 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:02.144 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:02.174 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-13b90cdd-79b7-4fd9-a4d2-1b573a739817 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:02.174 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-13b90cdd-79b7-4fd9-a4d2-1b573a739817 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:02.236 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-13b90cdd-79b7-4fd9-a4d2-1b573a739817 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:02.284 4076 3051055217304 MainThread INFO nova.compute.manager [req-13b90cdd-79b7-4fd9-a4d2-1b573a739817 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:03.000 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:52:03.000 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:03.000 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:03.035 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed vm state of instance-0000001a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:52:03.035 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed state of VM instance-0000001a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:52:03.112 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '4EACDAFB-0004-4104-9AEE-A86C4C9D1C41', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19973), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:03.156 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '7F5EA8FD-E7C5-4462-A4BC-D957158C9155', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20748), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:03.236 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job succeeded: {'InstanceID': '4EACDAFB-0004-4104-9AEE-A86C4C9D1C41', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30753), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:52:03.236 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:03.236 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Removing directory: C:\openstack\instances\instance-0000001a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:52:03.236 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:03.236 4076 3051055217128 MainThread ERROR nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Traceback (most recent call last): 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(*args, **kwargs) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] admin_password, network_info, block_device_info) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(self, *args, **kwds) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] should_plug_vifs=False) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'req_state': req_state}) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] time.sleep(sleep_time) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return f(*args, **kwargs) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return self._wait_for_job(job_path) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] error_desc=err_desc) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During handling of the above exception, another exception occurred: 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Traceback (most recent call last): 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] yield resources 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] accel_info=accel_info) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(*args, **kwargs) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] admin_password, network_info, block_device_info) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(self, *args, **kwds) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] should_plug_vifs=False) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'req_state': req_state}) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] time.sleep(sleep_time) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return f(*args, **kwargs) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return self._wait_for_job(job_path) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] error_desc=err_desc) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.236 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.251 4076 3051055217128 MainThread INFO nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Terminating instance 2021-09-10 16:52:03.251 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:52:03.251 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Got request to destroy instance 2021-09-10 16:52:03.281 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:52:03.281 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:03.289 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:03.289 4076 3051055217128 MainThread INFO nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:52:03.289 4076 3051055217128 MainThread DEBUG nova.compute.claims [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:52:03.289 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:03.317 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:03.317 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] VM Stopped (Lifecycle Event) 2021-09-10 16:52:03.329 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:03.329 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:03.329 4076 3051055217480 MainThread ERROR os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001b\' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C587810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)instance-0000001b7B7CCFFD-62EA-4DC7-9F49-D978D69C5878%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7F5EA8FD-E7C5-4462-A4BC-D957158C9155', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122861), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\nNot enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)\n\n'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878)", 'ErrorSummaryDescription': "'instance-0000001b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:03.329 4076 3051055217480 MainThread ERROR nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Failed to change vm state of instance-0000001b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. 2021-09-10 16:52:03.329 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Got request to destroy instance 2021-09-10 16:52:03.360 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e95057c8-d8a6-4790-8ab9-de0e85144333 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:03.360 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e95057c8-d8a6-4790-8ab9-de0e85144333 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:03.427 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e95057c8-d8a6-4790-8ab9-de0e85144333 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:03.485 4076 3051055217304 MainThread INFO nova.compute.manager [req-e95057c8-d8a6-4790-8ab9-de0e85144333 - - - - -] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:03.485 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:03.503 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:03.768 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.515s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:03.800 4076 3051055217128 MainThread ERROR nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Traceback (most recent call last): 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(*args, **kwargs) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] admin_password, network_info, block_device_info) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(self, *args, **kwds) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] should_plug_vifs=False) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'req_state': req_state}) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] time.sleep(sleep_time) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return f(*args, **kwargs) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return self._wait_for_job(job_path) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] error_desc=err_desc) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] During handling of the above exception, another exception occurred: 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Traceback (most recent call last): 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] accel_info=accel_info) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(*args, **kwargs) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] admin_password, network_info, block_device_info) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return function(self, *args, **kwds) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] should_plug_vifs=False) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'req_state': req_state}) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] time.sleep(sleep_time) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self.force_reraise() 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] raise self.value 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return f(*args, **kwargs) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] return self._wait_for_job(job_path) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] error_desc=err_desc) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. 2021-09-10 16:52:03.800 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Build of instance 7b215f4d-04c3-4607-aa80-c5dbc91487ee was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000001a' failed to start. Not enough memory in the system to start the virtual machine instance-0000001a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001a' failed to start. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Not enough memory in the system to start the virtual machine instance-0000001a with ram size 1024 megabytes. (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) 'instance-0000001a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6F9B5A07-5C7C-45E8-839D-AAC846045060) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:52:03.800 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:52:05.814 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:05.847 4076 3051055217128 MainThread INFO nova.compute.manager [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 7b215f4d-04c3-4607-aa80-c5dbc91487ee] Took 2.05 seconds to deallocate network for instance. 2021-09-10 16:52:05.866 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:52:05.866 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:05.866 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:05.896 4076 3051055217480 MainThread DEBUG os_win.utils.compute.vmutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Successfully changed vm state of instance-0000001b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:52:05.896 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Successfully changed state of VM instance-0000001b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:52:05.953 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Waiting for WMI job: {'InstanceID': '52D1C66E-6CC6-4DB0-830D-55C9C9C0AD56', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18771), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:06.157 4076 3051055217480 MainThread DEBUG os_win.utils.jobutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] WMI job succeeded: {'InstanceID': '52D1C66E-6CC6-4DB0-830D-55C9C9C0AD56', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=36953), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:52:06.157 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:06.157 4076 3051055217480 MainThread DEBUG os_win.utils.pathutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Removing directory: C:\openstack\instances\instance-0000001b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:52:06.157 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:06.157 4076 3051055217480 MainThread ERROR nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Traceback (most recent call last): 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(*args, **kwargs) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] admin_password, network_info, block_device_info) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(self, *args, **kwds) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] should_plug_vifs=False) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'req_state': req_state}) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] time.sleep(sleep_time) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return f(*args, **kwargs) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return self._wait_for_job(job_path) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] error_desc=err_desc) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During handling of the above exception, another exception occurred: 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Traceback (most recent call last): 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] yield resources 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] accel_info=accel_info) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(*args, **kwargs) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] admin_password, network_info, block_device_info) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(self, *args, **kwds) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] should_plug_vifs=False) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'req_state': req_state}) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] time.sleep(sleep_time) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return f(*args, **kwargs) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return self._wait_for_job(job_path) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] error_desc=err_desc) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.157 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.157 4076 3051055217480 MainThread INFO nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Terminating instance 2021-09-10 16:52:06.157 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:52:06.157 4076 3051055217480 MainThread INFO nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Got request to destroy instance 2021-09-10 16:52:06.191 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:52:06.191 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:06.204 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "instance-0000001b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:06.204 4076 3051055217480 MainThread INFO nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 16:52:06.204 4076 3051055217480 MainThread DEBUG nova.compute.claims [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:52:06.204 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:06.346 4076 3051055217128 MainThread INFO nova.scheduler.client.report [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Deleted allocations for instance 7b215f4d-04c3-4607-aa80-c5dbc91487ee 2021-09-10 16:52:06.424 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:06.453 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.hostops [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:06.516 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-e0c97b7c-8c71-4643-94bd-430318f14079 bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "7b215f4d-04c3-4607-aa80-c5dbc91487ee" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:06.703 4076 3051055217480 MainThread DEBUG nova.compute.provider_tree [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.scheduler.client.report [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:06.736 4076 3051055217480 MainThread ERROR nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Traceback (most recent call last): 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(*args, **kwargs) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] admin_password, network_info, block_device_info) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(self, *args, **kwds) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] should_plug_vifs=False) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'req_state': req_state}) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] time.sleep(sleep_time) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return f(*args, **kwargs) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return self._wait_for_job(job_path) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] error_desc=err_desc) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] During handling of the above exception, another exception occurred: 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Traceback (most recent call last): 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] accel_info=accel_info) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(*args, **kwargs) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] admin_password, network_info, block_device_info) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return function(self, *args, **kwds) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] should_plug_vifs=False) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'req_state': req_state}) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] time.sleep(sleep_time) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self.force_reraise() 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] raise self.value 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return f(*args, **kwargs) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] return self._wait_for_job(job_path) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] error_desc=err_desc) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. 2021-09-10 16:52:06.736 4076 3051055217480 MainThread TRACE nova.compute.manager [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.compute.utils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Build of instance f3ef8889-0aba-4544-aa33-a09b43d286aa was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000001b' failed to start. Not enough memory in the system to start the virtual machine instance-0000001b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001b' failed to start. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Not enough memory in the system to start the virtual machine instance-0000001b with ram size 1024 megabytes. (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) 'instance-0000001b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B7CCFFD-62EA-4DC7-9F49-D978D69C5878) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:52:06.736 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:52:08.002 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:08.032 4076 3051055217480 MainThread INFO nova.compute.manager [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: f3ef8889-0aba-4544-aa33-a09b43d286aa] Took 1.30 seconds to deallocate network for instance. 2021-09-10 16:52:08.392 4076 3051055217480 MainThread INFO nova.scheduler.client.report [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Deleted allocations for instance f3ef8889-0aba-4544-aa33-a09b43d286aa 2021-09-10 16:52:08.425 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-4242a0d8-ac56-43ee-97e5-64be11182803 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "f3ef8889-0aba-4544-aa33-a09b43d286aa" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 28.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:10.303 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:10.353 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:52:10.595 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:10.595 4076 3051093170488 MainThread DEBUG nova.virt.hardware [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:52:10.595 4076 3051093170488 MainThread INFO nova.compute.claims [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Claim successful on node n-h1-808474-1 2021-09-10 16:52:10.831 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:10.862 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:11.146 4076 3051093170488 MainThread DEBUG nova.compute.provider_tree [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:11.173 4076 3051093170488 MainThread DEBUG nova.scheduler.client.report [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:11.173 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.578s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:11.173 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:52:11.286 4076 3051093170488 MainThread DEBUG nova.compute.utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:52:11.286 4076 3051055217480 GreenThread-16 DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:52:11.286 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:52:11.316 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:52:11.578 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:52:11.578 4076 3051093170488 MainThread INFO nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Spawning new instance 2021-09-10 16:52:11.597 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:11.597 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:11.597 4076 3051093170488 MainThread DEBUG nova.block_device [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:52:11.597 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:11.612 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:11.612 4076 3051093170488 MainThread DEBUG os_win.utils.pathutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating directory: C:\openstack\instances\instance-0000001c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:52:11.612 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000001c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:52:11.657 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Resizing VHD C:\openstack\instances\instance-0000001c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:52:12.228 4076 3051055217480 GreenThread-16 DEBUG nova.policy [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bf4dce0601aa49c0b9ea6c48f2953557', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ae864ac350142778dcbcb12a8b3800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:52:14.548 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Successfully created port: d0f60e33-586c-4076-81c7-ad24460f18da _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:52:16.046 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Successfully updated port: d0f60e33-586c-4076-81c7-ad24460f18da _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:52:16.062 4076 3051055217480 GreenThread-16 DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Acquired lock "refresh_cache-5e428574-4db6-43d4-a54f-f0fe1bfd55e9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:52:16.062 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:52:16.281 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:52:16.959 4076 3051055217480 GreenThread-16 DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Updating instance_info_cache with network_info: [{"id": "d0f60e33-586c-4076-81c7-ad24460f18da", "address": "fa:16:3e:57:61:b8", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.44", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd0f60e33-58", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:16.983 4076 3051093169960 MainThread DEBUG nova.compute.manager [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Received event network-changed-d0f60e33-586c-4076-81c7-ad24460f18da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:52:16.983 4076 3051093169960 MainThread DEBUG nova.compute.manager [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Refreshing instance network info cache due to event network-changed-d0f60e33-586c-4076-81c7-ad24460f18da. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:52:16.999 4076 3051055217480 GreenThread-16 DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Releasing lock "refresh_cache-5e428574-4db6-43d4-a54f-f0fe1bfd55e9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:52:16.999 4076 3051055217480 GreenThread-16 DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Instance network_info: |[{"id": "d0f60e33-586c-4076-81c7-ad24460f18da", "address": "fa:16:3e:57:61:b8", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.44", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd0f60e33-58", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:52:16.999 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Preparing to wait for external event network-vif-plugged-d0f60e33-586c-4076-81c7-ad24460f18da prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:52:16.999 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:16.999 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:16.999 4076 3051093170488 MainThread DEBUG os_win.utils.compute.vmutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Creating VM instance-0000001c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:52:16.999 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-5e428574-4db6-43d4-a54f-f0fe1bfd55e9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:52:16.999 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Refreshing network info cache for port d0f60e33-586c-4076-81c7-ad24460f18da _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:52:17.937 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:52:18.315 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Updated VIF entry in instance network info cache for port d0f60e33-586c-4076-81c7-ad24460f18da. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:52:18.315 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Updating instance_info_cache with network_info: [{"id": "d0f60e33-586c-4076-81c7-ad24460f18da", "address": "fa:16:3e:57:61:b8", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.44", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd0f60e33-58", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:18.343 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-fd5aa486-5c02-46df-b73c-3c75fcd98d84 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-5e428574-4db6-43d4-a54f-f0fe1bfd55e9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:52:22.614 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-989da533-48ea-41a6-9a1d-f7729fb30335 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Received event network-vif-plugged-d0f60e33-586c-4076-81c7-ad24460f18da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:52:22.614 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-989da533-48ea-41a6-9a1d-f7729fb30335 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:22.614 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-989da533-48ea-41a6-9a1d-f7729fb30335 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:22.614 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-989da533-48ea-41a6-9a1d-f7729fb30335 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Processing event network-vif-plugged-d0f60e33-586c-4076-81c7-ad24460f18da _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:52:22.662 4076 3051093170488 MainThread INFO nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Using config drive for instance 2021-09-10 16:52:22.817 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:22.836 4076 3051093170488 MainThread INFO nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Creating config drive at C:\openstack\instances\instance-0000001c\configdrive.iso 2021-09-10 16:52:22.883 4076 3051093170488 MainThread DEBUG oslo_concurrency.processutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3dqs6yl9 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:52:23.234 4076 3051093170488 MainThread DEBUG oslo_concurrency.processutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3dqs6yl9" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:52:23.627 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Setting boot order for instance: instance-0000001c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:52:23.679 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:52:23.703 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '21B2D134-6357-4BA1-BE7C-E7266BB36072', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16574), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:23.861 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:23.861 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:23.861 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:23.861 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:23.879 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '21B2D134-6357-4BA1-BE7C-E7266BB36072', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=126517), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:23.879 4076 3051093170488 MainThread DEBUG os_win._utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.199798583984375 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:23.891 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-2bbc1fe7-724e-4313-b81b-2259a2c9406c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:23.891 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-2bbc1fe7-724e-4313-b81b-2259a2c9406c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:23.956 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-2bbc1fe7-724e-4313-b81b-2259a2c9406c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:23.985 4076 3051055217480 MainThread INFO nova.compute.manager [req-2bbc1fe7-724e-4313-b81b-2259a2c9406c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:24.906 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': 'DCA1D2A4-76E8-4419-8CED-AA21E893E9D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17144), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:25.000 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:25.067 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DCA1D2A4-76E8-4419-8CED-AA21E893E9D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=130334), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:25.067 4076 3051093170488 MainThread DEBUG os_win._utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3876030445098877 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:25.081 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:25.081 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:25.081 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:25.081 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:25.110 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-e298bfe6-6052-41ce-89c2-57d2a1687954 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:25.110 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-e298bfe6-6052-41ce-89c2-57d2a1687954 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:25.158 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-e298bfe6-6052-41ce-89c2-57d2a1687954 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:25.208 4076 3051055217480 MainThread INFO nova.compute.manager [req-e298bfe6-6052-41ce-89c2-57d2a1687954 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:25.793 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:25.793 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:52:26.033 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:52:26.033 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:52:26.099 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '3980FD59-F729-41A9-B673-F2953572953E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16826), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:26.284 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:26.284 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:26.284 4076 3051093169256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:26.284 4076 3051093169256 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:26.329 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-e3debb17-2c6f-4246-8f86-1d7f87c97f44 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:26.329 4076 3051093169256 MainThread DEBUG nova.virt.hyperv.vmops [req-e3debb17-2c6f-4246-8f86-1d7f87c97f44 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:26.360 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3980FD59-F729-41A9-B673-F2953572953E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=148893), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:26.360 4076 3051093170488 MainThread DEBUG os_win._utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.680765151977539 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:26.378 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-e3debb17-2c6f-4246-8f86-1d7f87c97f44 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:26.409 4076 3051093169256 MainThread INFO nova.compute.manager [req-e3debb17-2c6f-4246-8f86-1d7f87c97f44 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:27.079 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:27.117 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:52:27.117 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:52:27.117 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:27.117 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:27.126 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:27.126 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:27.174 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:27.174 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:27.174 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:52:27.174 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:27.203 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:27.408 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '8E61E246-99FA-4ECD-97F3-7CCBFB6A3CA6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=31930), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:27.507 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2866MB free_disk=52GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:52:27.507 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:27.601 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:27.601 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:27.601 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:27.601 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:27.703 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-1c094b63-cb12-4907-acad-4233c3ea75e1 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:27.703 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-1c094b63-cb12-4907-acad-4233c3ea75e1 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:27.703 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8E61E246-99FA-4ECD-97F3-7CCBFB6A3CA6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=165362), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:27.703 4076 3051093170488 MainThread DEBUG os_win._utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.023731708526611 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:27.766 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-1c094b63-cb12-4907-acad-4233c3ea75e1 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5e428574-4db6-43d4-a54f-f0fe1bfd55e9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:52:27.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:52:27.797 4076 3051055217480 MainThread INFO nova.compute.manager [req-1c094b63-cb12-4907-acad-4233c3ea75e1 - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:27.821 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:27.832 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:28.098 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:28.126 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:28.126 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:52:28.126 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:28.755 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '002D9FA7-D610-4B3A-9DF1-082C8BEED865', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15263), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:28.835 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:28.835 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:28.845 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:52:28.845 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:52:28.954 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:28.954 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:28.954 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:28.954 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:28.972 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-fc198c0e-eb01-4399-b843-ef436bf9a8df - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:28.972 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-fc198c0e-eb01-4399-b843-ef436bf9a8df - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:29.033 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '002D9FA7-D610-4B3A-9DF1-082C8BEED865', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=166761), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:29.033 4076 3051093170488 MainThread DEBUG os_win._utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.354045152664185 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:52:29.033 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-fc198c0e-eb01-4399-b843-ef436bf9a8df - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:29.112 4076 3051055217480 MainThread INFO nova.compute.manager [req-fc198c0e-eb01-4399-b843-ef436bf9a8df - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:30.062 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '520420E8-EE12-4FFF-A8B6-97AB74307673', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16903), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:30.218 4076 3051093170488 MainThread ERROR os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001c\' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)instance-0000001c480DF656-0D0E-4DC5-A768-D9944BD2376A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '520420E8-EE12-4FFF-A8B6-97AB74307673', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=136167), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 52, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 52, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\nNot enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)\n\n'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A)", 'ErrorSummaryDescription': "'instance-0000001c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:52:30.235 4076 3051093170488 MainThread ERROR nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Failed to change vm state of instance-0000001c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. 2021-09-10 16:52:30.235 4076 3051093170488 MainThread INFO nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Got request to destroy instance 2021-09-10 16:52:30.235 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:30.235 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:30.235 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:52:30.235 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] VM Stopped (Lifecycle Event) 2021-09-10 16:52:30.250 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-f3f7adfc-dcc7-49e2-acd5-76b1adcda13c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:30.250 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-f3f7adfc-dcc7-49e2-acd5-76b1adcda13c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:30.334 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-f3f7adfc-dcc7-49e2-acd5-76b1adcda13c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:52:30.359 4076 3051055217480 MainThread INFO nova.compute.manager [req-f3f7adfc-dcc7-49e2-acd5-76b1adcda13c - - - - -] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:52:32.375 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:52:32.375 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:32.375 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:32.390 4076 3051093170488 MainThread DEBUG os_win.utils.compute.vmutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed vm state of instance-0000001c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:52:32.390 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Successfully changed state of VM instance-0000001c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:52:32.454 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Waiting for WMI job: {'InstanceID': '4D0AF92A-9204-47DB-B75F-8458D4DF8F1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20732), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:32.563 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] WMI job succeeded: {'InstanceID': '4D0AF92A-9204-47DB-B75F-8458D4DF8F1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=39917), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:52:32.563 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:32.563 4076 3051093170488 MainThread DEBUG os_win.utils.pathutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Removing directory: C:\openstack\instances\instance-0000001c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:52:32.563 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:32.581 4076 3051093170488 MainThread ERROR nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Traceback (most recent call last): 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(*args, **kwargs) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] admin_password, network_info, block_device_info) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(self, *args, **kwds) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] should_plug_vifs=False) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'req_state': req_state}) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] time.sleep(sleep_time) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return f(*args, **kwargs) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return self._wait_for_job(job_path) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] error_desc=err_desc) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During handling of the above exception, another exception occurred: 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Traceback (most recent call last): 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] yield resources 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] accel_info=accel_info) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(*args, **kwargs) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] admin_password, network_info, block_device_info) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(self, *args, **kwds) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] should_plug_vifs=False) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'req_state': req_state}) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] time.sleep(sleep_time) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return f(*args, **kwargs) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return self._wait_for_job(job_path) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] error_desc=err_desc) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:32.581 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:32.581 4076 3051093170488 MainThread INFO nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Terminating instance 2021-09-10 16:52:32.581 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:52:32.581 4076 3051093170488 MainThread INFO nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Got request to destroy instance 2021-09-10 16:52:32.593 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:52:32.593 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:32.593 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "instance-0000001c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:32.593 4076 3051093170488 MainThread INFO nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:52:32.593 4076 3051093170488 MainThread DEBUG nova.compute.claims [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:52:32.593 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:32.784 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:32.812 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:33.039 4076 3051093170488 MainThread DEBUG nova.compute.provider_tree [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:33.064 4076 3051093170488 MainThread DEBUG nova.scheduler.client.report [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:33.064 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:33.064 4076 3051093170488 MainThread ERROR nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Traceback (most recent call last): 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(*args, **kwargs) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] admin_password, network_info, block_device_info) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(self, *args, **kwds) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] should_plug_vifs=False) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'req_state': req_state}) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] time.sleep(sleep_time) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return f(*args, **kwargs) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return self._wait_for_job(job_path) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] error_desc=err_desc) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] During handling of the above exception, another exception occurred: 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Traceback (most recent call last): 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] accel_info=accel_info) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(*args, **kwargs) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] admin_password, network_info, block_device_info) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return function(self, *args, **kwds) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] should_plug_vifs=False) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'req_state': req_state}) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] time.sleep(sleep_time) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self.force_reraise() 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] raise self.value 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return f(*args, **kwargs) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] return self._wait_for_job(job_path) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] error_desc=err_desc) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. 2021-09-10 16:52:33.064 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] 2021-09-10 16:52:33.064 4076 3051093170488 MainThread DEBUG nova.compute.utils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:52:33.064 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Build of instance 5e428574-4db6-43d4-a54f-f0fe1bfd55e9 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000001c' failed to start. Not enough memory in the system to start the virtual machine instance-0000001c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001c' failed to start. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Not enough memory in the system to start the virtual machine instance-0000001c with ram size 1024 megabytes. (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) 'instance-0000001c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 480DF656-0D0E-4DC5-A768-D9944BD2376A) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:52:33.064 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:52:33.078 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:52:33.078 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:52:33.078 4076 3051093170488 MainThread DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:52:34.971 4076 3051093170488 MainThread DEBUG nova.network.neutron [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:35.002 4076 3051093170488 MainThread INFO nova.compute.manager [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 5e428574-4db6-43d4-a54f-f0fe1bfd55e9] Took 1.92 seconds to deallocate network for instance. 2021-09-10 16:52:35.377 4076 3051093170488 MainThread INFO nova.scheduler.client.report [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Deleted allocations for instance 5e428574-4db6-43d4-a54f-f0fe1bfd55e9 2021-09-10 16:52:35.421 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-bdd82277-b513-42d2-949c-a4893874c11d bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "5e428574-4db6-43d4-a54f-f0fe1bfd55e9" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:46.190 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:46.190 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:46.190 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:46.190 4076 3051093170488 MainThread ERROR nova.compute.manager [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Traceback (most recent call last): 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._delete_instance(context, instance, bdms) 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] NameError: name 'instant' is not defined 2021-09-10 16:52:46.190 4076 3051093170488 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] 2021-09-10 16:52:46.296 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.109s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:46.467 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:46.505 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:46.527 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.hostops [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:46.723 4076 3051093170488 MainThread DEBUG nova.compute.provider_tree [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:46.723 4076 3051093170488 MainThread DEBUG nova.scheduler.client.report [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:46.723 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:46.723 4076 3051093170488 MainThread INFO nova.compute.manager [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:52:46.749 4076 3051093170488 MainThread ERROR oslo_messaging.rpc.server [req-3c9c4b14-0eb2-4cdb-b05f-1cd23941b0b9 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:52:46.749 4076 3051093170488 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:52:47.393 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:52:47.393 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:52:47.568 4076 3051093170488 MainThread INFO nova.compute.manager [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] instance snapshotting 2021-09-10 16:52:47.568 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating snapshot for instance instance-00000018 snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:50 2021-09-10 16:52:47.680 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': 'A7326518-08E8-4B0D-90C2-524EEE5F1AFB', 'Description': 'Creating Virtual Machine Checkpoint', 'ElementName': 'Creating Checkpoint', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=25105), 'Cancellable': True, 'JobType': 70, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 7}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:52:48.562 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job succeeded: {'InstanceID': 'A7326518-08E8-4B0D-90C2-524EEE5F1AFB', 'Description': 'Creating Virtual Machine Checkpoint', 'ElementName': 'Creating Checkpoint', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=905391), 'Cancellable': True, 'JobType': 70, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:52:48.735 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Getting info for VHD C:\openstack\instances\instance-00000018\root.vhdx snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:59 2021-09-10 16:52:48.760 4076 3051093170488 MainThread DEBUG os_win.utils.pathutils [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Creating directory: C:\openstack\instances\export\instance-00000018 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:52:48.760 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Copying VHD C:\openstack\instances\instance-00000018\root.vhdx to C:\openstack\instances\export\instance-00000018\root.vhdx snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:69 2021-09-10 16:52:48.844 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Copying base disk C:\openstack\instances\instance-00000018\root.vhdx to C:\openstack\instances\export\instance-00000018\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:81 2021-09-10 16:52:49.910 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Reconnecting copied base VHD C:\openstack\instances\export\instance-00000018\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx and diff VHD C:\openstack\instances\export\instance-00000018\root.vhdx snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:89 2021-09-10 16:52:50.296 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Merging diff disk C:\openstack\instances\export\instance-00000018\root.vhdx into its parent. snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:94 2021-09-10 16:52:50.422 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Updating Glance image 45681f94-8ce0-497c-aefb-0ecc624b337d with content from merged disk C:\openstack\instances\export\instance-00000018\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:100 2021-09-10 16:52:52.299 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-7fbb2fe1-61a5-4429-acf2-825dc5d1e429 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received event network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:52:52.299 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-7fbb2fe1-61a5-4429-acf2-825dc5d1e429 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:52.299 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-7fbb2fe1-61a5-4429-acf2-825dc5d1e429 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:52.299 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-7fbb2fe1-61a5-4429-acf2-825dc5d1e429 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] No waiting events found dispatching network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:52:52.299 4076 3051093172072 MainThread WARNING nova.compute.manager [req-7fbb2fe1-61a5-4429-acf2-825dc5d1e429 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received unexpected event network-vif-plugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb for instance with vm_state active and task_state image_uploading. 2021-09-10 16:52:53.484 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:53.484 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:53.484 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:53.484 4076 3051093172072 MainThread ERROR nova.compute.manager [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Traceback (most recent call last): 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._delete_instance(context, instance, bdms) 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] NameError: name 'instant' is not defined 2021-09-10 16:52:53.484 4076 3051093172072 MainThread TRACE nova.compute.manager [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] 2021-09-10 16:52:53.681 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:53.871 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:52:53.881 4076 3051093172072 MainThread DEBUG nova.virt.hyperv.hostops [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:52:53.897 4076 3051093172072 MainThread DEBUG nova.virt.hyperv.hostops [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:52:54.145 4076 3051093172072 MainThread DEBUG nova.compute.provider_tree [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:52:54.158 4076 3051093172072 MainThread DEBUG nova.scheduler.client.report [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:52:54.158 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.281s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:52:54.158 4076 3051093172072 MainThread INFO nova.compute.manager [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:52:54.174 4076 3051093172072 MainThread ERROR oslo_messaging.rpc.server [req-5a692a96-cf43-47fc-92ae-1c6520de0a63 66161d18b31e45309aee94f739bc6a3f 145f8177efc74f5e9be31c5f74e69d0a - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:52:54.174 4076 3051093172072 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:52:54.861 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Received event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:52:54.861 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing instance network info cache due to event network-changed-df65fbf2-3d95-4273-adfa-5d65cf85d7c1. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:52:54.861 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:52:54.861 4076 3051093172072 MainThread DEBUG nova.network.neutron [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Refreshing network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:52:56.658 4076 3051093172072 MainThread DEBUG nova.network.neutron [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated VIF entry in instance network info cache for port df65fbf2-3d95-4273-adfa-5d65cf85d7c1. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:52:56.658 4076 3051093172072 MainThread DEBUG nova.network.neutron [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:52:56.707 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [req-4dde38f9-dffc-452f-ae44-aabb7ae80fc6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:16.342 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Snapshot image 45681f94-8ce0-497c-aefb-0ecc624b337d updated for VM instance-00000018 snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:107 2021-09-10 16:53:16.342 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Removing snapshot 45681f94-8ce0-497c-aefb-0ecc624b337d snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:110 2021-09-10 16:53:16.389 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Waiting for WMI job: {'InstanceID': '828EE773-AB28-49CB-B8FA-84196E54665F', 'Description': 'Deleting Checkpoint', 'ElementName': 'Deleting Checkpoint', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19210), 'Cancellable': False, 'JobType': 72, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:16.505 4076 3051093170488 MainThread DEBUG os_win.utils.jobutils [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] WMI job succeeded: {'InstanceID': '828EE773-AB28-49CB-B8FA-84196E54665F', 'Description': 'Deleting Checkpoint', 'ElementName': 'Deleting Checkpoint', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=50743), 'Cancellable': False, 'JobType': 72, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:53:16.505 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.snapshotops [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Removing directory: C:\openstack\instances\export\instance-00000018 snapshot c:\openstack\build\nova\nova\virt\hyperv\snapshotops.py:116 2021-09-10 16:53:16.607 4076 3051093170488 MainThread INFO nova.compute.manager [req-832f53cb-b103-4c86-8389-f675171a8c2d 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Took 29.05 seconds to snapshot the instance on the hypervisor. 2021-09-10 16:53:17.486 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:17.509 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:53:17.612 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:17.612 4076 3051054567848 MainThread DEBUG nova.virt.hardware [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:53:17.612 4076 3051054567848 MainThread INFO nova.compute.claims [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Claim successful on node n-h1-808474-1 2021-09-10 16:53:17.748 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:17.755 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:17.958 4076 3051054567848 MainThread DEBUG nova.compute.provider_tree [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:17.971 4076 3051054567848 MainThread DEBUG nova.scheduler.client.report [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:17.971 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:17.971 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:53:18.065 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:53:18.065 4076 3051054567672 GreenThread-17 DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:53:18.065 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:53:18.134 4076 3051054567848 MainThread INFO nova.virt.block_device [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Booting with volume a0ac4351-dd95-40e6-bcc7-f7e316d40088 at /dev/sda 2021-09-10 16:53:18.217 4076 3051054567848 MainThread DEBUG os_brick.utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:18.217 4076 3051054567848 MainThread DEBUG os_brick.utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:18.217 4076 3051054567848 MainThread DEBUG nova.virt.block_device [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Updating existing volume attachment record: 48ddfaa5-b2ae-43ca-84e1-d02ec5bcd065 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:53:18.366 4076 3051054567672 GreenThread-17 DEBUG nova.policy [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'fac7f2174628404cb8576928225b4394', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fa8236f1477243dfafd0682b608fc659', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:53:18.725 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:18.756 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:53:18.908 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:18.908 4076 3051054569608 MainThread DEBUG nova.virt.hardware [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:53:18.908 4076 3051054569608 MainThread INFO nova.compute.claims [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Claim successful on node n-h1-808474-1 2021-09-10 16:53:19.029 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.hostops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:19.047 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.hostops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:19.250 4076 3051054569432 MainThread DEBUG nova.compute.manager [req-b249fdc6-a12c-4965-9c50-e5e87ad098de 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-vif-deleted-4976e995-8bf8-4ddb-833a-7cc8f525682f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:19.250 4076 3051054569608 MainThread DEBUG nova.compute.provider_tree [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:19.264 4076 3051054569608 MainThread DEBUG nova.scheduler.client.report [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:19.264 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:19.264 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:53:19.342 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:53:19.366 4076 3051054569432 GreenThread-18 DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:53:19.366 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:53:19.505 4076 3051054569432 GreenThread-18 DEBUG nova.policy [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'beffe9ba8529426fbb27e549254bd679', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91237ebda1b54d2dbc761e48fdd93516', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:53:19.553 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:53:19.553 4076 3051054569608 MainThread INFO nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Spawning new instance 2021-09-10 16:53:19.577 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-0000001f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:19.577 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-0000001f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:19.577 4076 3051054569608 MainThread DEBUG nova.block_device [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:53:19.577 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:19.577 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:19.605 4076 3051054569608 MainThread DEBUG os_win.utils.pathutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating directory: C:\openstack\instances\instance-0000001f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:53:19.605 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000001f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:53:19.634 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Resizing VHD C:\openstack\instances\instance-0000001f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:53:19.874 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Successfully created port: 76db5559-28e4-4b26-bfce-714d72b14bfd _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:53:20.374 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Successfully created port: 5beac292-790e-4830-8f59-e411516f08cc _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:53:21.029 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:21.225 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Successfully updated port: 5beac292-790e-4830-8f59-e411516f08cc _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:53:21.241 4076 3051054569432 GreenThread-18 DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:21.241 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:53:21.325 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:53:21.673 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Successfully updated port: 76db5559-28e4-4b26-bfce-714d72b14bfd _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:53:21.702 4076 3051054567672 GreenThread-17 DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Acquired lock "refresh_cache-079b5c5d-fe33-48d2-ba84-de08229901eb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:21.702 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:53:21.736 4076 3051054569432 GreenThread-18 DEBUG nova.network.neutron [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:21.756 4076 3051054569432 GreenThread-18 DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:21.756 4076 3051054569432 GreenThread-18 DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Instance network_info: |[{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:53:21.756 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Preparing to wait for external event network-vif-plugged-5beac292-790e-4830-8f59-e411516f08cc prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:53:21.756 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:21.756 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:21.756 4076 3051054569608 MainThread DEBUG os_win.utils.compute.vmutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating VM instance-0000001f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:53:21.845 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:53:22.581 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:53:22.629 4076 3051054567672 GreenThread-17 DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Updating instance_info_cache with network_info: [{"id": "76db5559-28e4-4b26-bfce-714d72b14bfd", "address": "fa:16:3e:c2:85:3e", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap76db5559-28", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:22.664 4076 3051054567672 GreenThread-17 DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Releasing lock "refresh_cache-079b5c5d-fe33-48d2-ba84-de08229901eb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:22.664 4076 3051054567672 GreenThread-17 DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Instance network_info: |[{"id": "76db5559-28e4-4b26-bfce-714d72b14bfd", "address": "fa:16:3e:c2:85:3e", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap76db5559-28", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:53:22.737 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-changed-5beac292-790e-4830-8f59-e411516f08cc external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:22.737 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing instance network info cache due to event network-changed-5beac292-790e-4830-8f59-e411516f08cc. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:53:22.737 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:22.737 4076 3051054570312 MainThread DEBUG nova.network.neutron [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing network info cache for port 5beac292-790e-4830-8f59-e411516f08cc _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:53:22.788 4076 3051054569432 MainThread DEBUG nova.compute.manager [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Received event network-changed-76db5559-28e4-4b26-bfce-714d72b14bfd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:22.788 4076 3051054569432 MainThread DEBUG nova.compute.manager [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Refreshing instance network info cache due to event network-changed-76db5559-28e4-4b26-bfce-714d72b14bfd. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:53:22.788 4076 3051054569432 MainThread DEBUG oslo_concurrency.lockutils [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-079b5c5d-fe33-48d2-ba84-de08229901eb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:22.788 4076 3051054569432 MainThread DEBUG nova.network.neutron [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Refreshing network info cache for port 76db5559-28e4-4b26-bfce-714d72b14bfd _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:53:23.052 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:53:23.052 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Spawning new instance 2021-09-10 16:53:23.052 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:23.052 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:23.076 4076 3051054567848 MainThread DEBUG nova.block_device [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:53:23.076 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Preparing to wait for external event network-vif-plugged-76db5559-28e4-4b26-bfce-714d72b14bfd prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:53:23.076 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:23.076 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:23.076 4076 3051054567848 MainThread DEBUG os_win.utils.compute.vmutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Creating VM instance-0000001d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:53:23.164 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:23.164 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:23.164 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:23.164 4076 3051054567672 MainThread ERROR nova.compute.manager [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Traceback (most recent call last): 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] self._delete_instance(context, instance, bdms) 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] NameError: name 'instant' is not defined 2021-09-10 16:53:23.164 4076 3051054567672 MainThread TRACE nova.compute.manager [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] 2021-09-10 16:53:23.220 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:23.318 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:23.346 4076 3051054567672 MainThread DEBUG nova.virt.hyperv.hostops [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:23.346 4076 3051054567672 MainThread DEBUG nova.virt.hyperv.hostops [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:23.373 4076 3051054569432 MainThread DEBUG nova.network.neutron [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Updated VIF entry in instance network info cache for port 76db5559-28e4-4b26-bfce-714d72b14bfd. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:53:23.373 4076 3051054569432 MainThread DEBUG nova.network.neutron [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Updating instance_info_cache with network_info: [{"id": "76db5559-28e4-4b26-bfce-714d72b14bfd", "address": "fa:16:3e:c2:85:3e", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap76db5559-28", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:23.390 4076 3051054569432 MainThread DEBUG oslo_concurrency.lockutils [req-b48508cf-48e4-46cc-9b77-7534bfb5359a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-079b5c5d-fe33-48d2-ba84-de08229901eb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:23.545 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.volumeops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'target_lun': 0, 'volume_id': 'a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'auth_method': 'CHAP', 'auth_username': 'WUVTEq8Na5sjUfeUS977', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '079b5c5d-fe33-48d2-ba84-de08229901eb', 'attached_at': '', 'detached_at': '', 'volume_id': 'a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'serial': 'a0ac4351-dd95-40e6-bcc7-f7e316d40088'} to instance-0000001d _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:53:23.545 4076 3051054567848 MainThread DEBUG os_brick.initiator.connector [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Factory for iscsi on None factory c:\python37\lib\site-packages\os_brick\initiator\connector.py:282 2021-09-10 16:53:23.545 4076 3051054567848 MainThread INFO os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] No iSCSI initiator was explicitly requested. The Microsoft iSCSI initiator will choose the initiator when establishing sessions. 2021-09-10 16:53:23.545 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'target_lun': 0, 'volume_id': 'a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'auth_method': 'CHAP', 'auth_username': 'WUVTEq8Na5sjUfeUS977', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:23.545 4076 3051054567848 MainThread INFO os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:53:23.545 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:53:23.579 4076 3051054567672 MainThread DEBUG nova.compute.provider_tree [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:23.579 4076 3051054567672 MainThread DEBUG nova.scheduler.client.report [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:23.579 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:23.579 4076 3051054567672 MainThread INFO nova.compute.manager [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:53:23.579 4076 3051054567672 MainThread ERROR oslo_messaging.rpc.server [req-4d0ff9ca-5334-48c7-b3e3-c8a77cb3d80f bf4dce0601aa49c0b9ea6c48f2953557 3ae864ac350142778dcbcb12a8b3800e - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:53:23.579 4076 3051054567672 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:53:23.881 4076 3051054570312 MainThread DEBUG nova.network.neutron [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated VIF entry in instance network info cache for port 5beac292-790e-4830-8f59-e411516f08cc. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:53:23.881 4076 3051054570312 MainThread DEBUG nova.network.neutron [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:23.896 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-c7fb6b5c-eba0-411e-aa8b-2b256e36a0c8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:24.435 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-98e8dbe1-1263-4a73-918d-3150bd727179 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received event network-vif-unplugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:24.435 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-98e8dbe1-1263-4a73-918d-3150bd727179 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:24.446 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-98e8dbe1-1263-4a73-918d-3150bd727179 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:24.446 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-98e8dbe1-1263-4a73-918d-3150bd727179 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] No waiting events found dispatching network-vif-unplugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:53:24.446 4076 3051054570312 MainThread WARNING nova.compute.manager [req-98e8dbe1-1263-4a73-918d-3150bd727179 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Received unexpected event network-vif-unplugged-0110d67d-01ad-4b98-9caf-80b01bc70bfb for instance with vm_state active and task_state None. 2021-09-10 16:53:24.812 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:26.496 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-7518abde-bf85-44aa-9ec0-440c695dd9ab 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-vif-plugged-5beac292-790e-4830-8f59-e411516f08cc external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:26.496 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-7518abde-bf85-44aa-9ec0-440c695dd9ab 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:26.496 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [req-7518abde-bf85-44aa-9ec0-440c695dd9ab 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:26.496 4076 3051054570312 MainThread DEBUG nova.compute.manager [req-7518abde-bf85-44aa-9ec0-440c695dd9ab 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Processing event network-vif-plugged-5beac292-790e-4830-8f59-e411516f08cc _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:53:26.530 4076 3051054569608 MainThread INFO nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Using config drive for instance 2021-09-10 16:53:26.689 4076 3051054569608 MainThread INFO nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Creating config drive at C:\openstack\instances\instance-0000001f\configdrive.iso 2021-09-10 16:53:26.719 4076 3051054569608 MainThread DEBUG oslo_concurrency.processutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu276dssg execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:53:26.849 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:26.873 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:27.030 4076 3051054569608 MainThread DEBUG oslo_concurrency.processutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu276dssg" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:53:27.410 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Setting boot order for instance: instance-0000001f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:53:27.453 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:53:27.498 4076 3051054569608 MainThread DEBUG os_win.utils.jobutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': '926EA258-EDB0-4A76-B87F-65C06455FD1D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15046), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:27.773 4076 3051054567672 MainThread DEBUG nova.virt.driver [-] Emitting event Started> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:27.773 4076 3051054567672 MainThread INFO nova.compute.manager [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] VM Started (Lifecycle Event) 2021-09-10 16:53:27.773 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:27.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:27.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:53:27.791 4076 3051054567672 MainThread DEBUG nova.compute.manager [req-ab31e730-f745-4826-9aa9-c42558f2a068 - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:27.791 4076 3051054567672 MainThread DEBUG nova.virt.hyperv.vmops [req-ab31e730-f745-4826-9aa9-c42558f2a068 - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:27.812 4076 3051054570312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:27.842 4076 3051054567672 MainThread DEBUG nova.compute.manager [req-ab31e730-f745-4826-9aa9-c42558f2a068 - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Synchronizing instance power state after lifecycle event "Started"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 1 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:27.858 4076 3051054569608 MainThread DEBUG os_win.utils.jobutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job succeeded: {'InstanceID': '926EA258-EDB0-4A76-B87F-65C06455FD1D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=269213), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:53:27.858 4076 3051054569608 MainThread DEBUG os_win.utils.compute.vmutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed vm state of instance-0000001f to 2 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:53:27.858 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed state of VM instance-0000001f to: 2 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:53:27.858 4076 3051054569608 MainThread INFO nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Took 8.31 seconds to spawn the instance on the hypervisor. 2021-09-10 16:53:27.858 4076 3051054569608 MainThread DEBUG nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:27.858 4076 3051054569608 MainThread DEBUG nova.virt.hyperv.vmops [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:27.879 4076 3051054567672 MainThread INFO nova.compute.manager [req-ab31e730-f745-4826-9aa9-c42558f2a068 - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:27.997 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:27.997 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:53:28.013 4076 3051054569608 MainThread INFO nova.compute.manager [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Took 9.16 seconds to build instance. 2021-09-10 16:53:28.056 4076 3051054569608 MainThread DEBUG oslo_concurrency.lockutils [req-db57e51c-ac83-40a9-bbcb-3db0152ae027 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 9.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:28.583 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:53:28.725 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:53:28.773 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] <== connect_volume: return (5227ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '36001405F29D523C8D7247F8B884E7E52'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:29.047 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:29.063 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:29.063 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:53:29.063 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:29.063 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:29.081 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:29.081 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:29.113 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:29.113 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:29.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:53:29.113 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:29.141 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:29.210 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:53:29.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2126MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:53:29.362 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 079b5c5d-fe33-48d2-ba84-de08229901eb actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.522 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:53:29.545 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:53:29.569 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:29.593 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:29.797 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:29.813 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:29.813 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:53:29.813 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:31.061 4076 3051054570136 MainThread DEBUG oslo_concurrency.lockutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name..do_reserve" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:31.078 4076 3051054570136 MainThread DEBUG nova.compute.utils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Using /dev/sd instead of /dev/xvd get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:53:31.096 4076 3051054570136 MainThread DEBUG oslo_concurrency.lockutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager.reserve_block_device_name..do_reserve" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:31.225 4076 3051054570136 MainThread DEBUG oslo_concurrency.lockutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager.attach_volume..do_attach_volume" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:31.225 4076 3051054570136 MainThread INFO nova.compute.manager [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Attaching volume 6fe38b5b-f580-4b49-8822-02f1519e6936 to /dev/sdb 2021-09-10 16:53:31.303 4076 3051054570136 MainThread DEBUG os_brick.utils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:31.303 4076 3051054570136 MainThread DEBUG os_brick.utils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:31.315 4076 3051054570136 MainThread DEBUG nova.virt.block_device [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating existing volume attachment record: db68e464-635a-43fc-81e3-e6681a734234 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:53:31.529 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:53:31.529 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:53:34.248 4076 3051054567672 MainThread DEBUG nova.compute.manager [req-c0d92ea9-08f1-43e9-8b4e-379fe335b6d7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Received event network-vif-plugged-76db5559-28e4-4b26-bfce-714d72b14bfd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:34.248 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-c0d92ea9-08f1-43e9-8b4e-379fe335b6d7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:34.248 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [req-c0d92ea9-08f1-43e9-8b4e-379fe335b6d7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:34.248 4076 3051054567672 MainThread DEBUG nova.compute.manager [req-c0d92ea9-08f1-43e9-8b4e-379fe335b6d7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Processing event network-vif-plugged-76db5559-28e4-4b26-bfce-714d72b14bfd _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:53:34.297 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Using config drive for instance 2021-09-10 16:53:35.345 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Creating config drive at C:\openstack\instances\instance-0000001d\configdrive.iso 2021-09-10 16:53:35.373 4076 3051054567848 MainThread DEBUG oslo_concurrency.processutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy4cwuiq4 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:53:35.702 4076 3051054567848 MainThread DEBUG oslo_concurrency.processutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000001d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy4cwuiq4" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:53:36.108 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Setting boot order for instance: instance-0000001d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:53:36.164 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:53:36.193 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': 'CE820CA0-4CC3-4B59-A60E-D2F880849726', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16071), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:36.346 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CE820CA0-4CC3-4B59-A60E-D2F880849726', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124487), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:36.346 4076 3051054567848 MainThread DEBUG os_win._utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18269681930541992 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:36.346 4076 3051054568376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:36.346 4076 3051054568376 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:36.346 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:36.346 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:36.381 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-f9a964c4-b2d7-4b6a-920d-1da8b2578220 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:36.381 4076 3051054568376 MainThread DEBUG nova.virt.hyperv.vmops [req-f9a964c4-b2d7-4b6a-920d-1da8b2578220 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:36.435 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-f9a964c4-b2d7-4b6a-920d-1da8b2578220 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:36.471 4076 3051054568376 MainThread INFO nova.compute.manager [req-f9a964c4-b2d7-4b6a-920d-1da8b2578220 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:37.374 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': 'D8FE11C9-1714-4577-8FE9-77632DE0C56C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14873), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:37.546 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:37.546 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:37.546 4076 3051054568376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:37.546 4076 3051054568376 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:37.576 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-f0e5ded0-d0a6-4ec8-af01-b3f74f43963a - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:37.576 4076 3051054568376 MainThread DEBUG nova.virt.hyperv.vmops [req-f0e5ded0-d0a6-4ec8-af01-b3f74f43963a - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:37.627 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D8FE11C9-1714-4577-8FE9-77632DE0C56C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=141529), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:37.627 4076 3051054567848 MainThread DEBUG os_win._utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4636986255645752 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:37.627 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-f0e5ded0-d0a6-4ec8-af01-b3f74f43963a - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:37.655 4076 3051054568376 MainThread INFO nova.compute.manager [req-f0e5ded0-d0a6-4ec8-af01-b3f74f43963a - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:38.656 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': '6CBEBE59-4074-4341-8F79-F51F3FFBFCE4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14609), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:38.812 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:38.812 4076 3051054567672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:38.812 4076 3051054568376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:38.812 4076 3051054568376 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:38.812 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6CBEBE59-4074-4341-8F79-F51F3FFBFCE4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=132636), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:38.812 4076 3051054567848 MainThread DEBUG os_win._utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.648491382598877 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:38.905 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-66782506-9875-4dd1-9614-7caff955dfd8 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:38.932 4076 3051054568376 MainThread DEBUG nova.virt.hyperv.vmops [req-66782506-9875-4dd1-9614-7caff955dfd8 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:38.968 4076 3051054568376 MainThread DEBUG nova.compute.manager [req-66782506-9875-4dd1-9614-7caff955dfd8 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:39.014 4076 3051054568376 MainThread INFO nova.compute.manager [req-66782506-9875-4dd1-9614-7caff955dfd8 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:39.139 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:39.154 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:53:39.326 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:39.350 4076 3051054569256 MainThread DEBUG nova.virt.hardware [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:53:39.350 4076 3051054569256 MainThread INFO nova.compute.claims [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Claim successful on node n-h1-808474-1 2021-09-10 16:53:39.576 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:39.596 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:39.796 4076 3051054569256 MainThread DEBUG nova.compute.provider_tree [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:39.827 4076 3051054569256 MainThread DEBUG nova.scheduler.client.report [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:39.827 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:39.827 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:53:39.857 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': '20F667A6-EBF0-4D49-968E-A0E5B955599B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23299), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:39.968 4076 3051054569080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:39.968 4076 3051054569080 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:39.968 4076 3051054571016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:39.995 4076 3051054571016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:40.010 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '20F667A6-EBF0-4D49-968E-A0E5B955599B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118421), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:40.010 4076 3051054567848 MainThread DEBUG os_win._utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8464255332946777 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:40.029 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:53:40.037 4076 3051054571016 GreenThread-19 DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:53:40.037 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:53:40.057 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-741c491c-b401-422a-bf39-00793962f34f - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:40.057 4076 3051054569080 MainThread DEBUG nova.virt.hyperv.vmops [req-741c491c-b401-422a-bf39-00793962f34f - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:40.115 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-741c491c-b401-422a-bf39-00793962f34f - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:40.157 4076 3051054569080 MainThread INFO nova.compute.manager [req-741c491c-b401-422a-bf39-00793962f34f - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:40.312 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:53:40.334 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Spawning new instance 2021-09-10 16:53:40.334 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:40.334 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:40.334 4076 3051054569256 MainThread DEBUG nova.block_device [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:53:40.334 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:40.352 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:40.358 4076 3051054569256 MainThread DEBUG os_win.utils.pathutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating directory: C:\openstack\instances\instance-00000021 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:53:40.358 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000021\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:53:40.382 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Resizing VHD C:\openstack\instances\instance-00000021\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:53:40.435 4076 3051054571016 GreenThread-19 DEBUG nova.policy [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd33d11497a91483d93fe0ed8f5c26603', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'efc5deaa89294889821f303976e2d776', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:53:41.048 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': '897F934A-3271-4790-A027-1CCC03097FD4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15442), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:41.189 4076 3051054569080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:41.189 4076 3051054569080 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:41.189 4076 3051054570488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:41.189 4076 3051054570488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:41.189 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '897F934A-3271-4790-A027-1CCC03097FD4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120250), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:41.206 4076 3051054567848 MainThread DEBUG os_win._utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.04276967048645 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:41.206 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-0167d087-d9c3-487d-9f85-d33b8413d2ef - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:41.206 4076 3051054569080 MainThread DEBUG nova.virt.hyperv.vmops [req-0167d087-d9c3-487d-9f85-d33b8413d2ef - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:41.273 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-0167d087-d9c3-487d-9f85-d33b8413d2ef - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:41.420 4076 3051054569080 MainThread INFO nova.compute.manager [req-0167d087-d9c3-487d-9f85-d33b8413d2ef - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:42.237 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': '27060891-67CB-4037-A7C6-9A563AEA261D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15803), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:42.382 4076 3051054570488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:42.382 4076 3051054570488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:42.382 4076 3051054569080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:42.382 4076 3051054569080 MainThread INFO nova.compute.manager [-] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] VM Stopped (Lifecycle Event) 2021-09-10 16:53:42.392 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000001d\' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C4010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000001d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)instance-0000001dB930DB5B-44E5-40E4-8ED1-1572105A0C40%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '27060891-67CB-4037-A7C6-9A563AEA261D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117406), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\nNot enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)\n\n'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40)", 'ErrorSummaryDescription': "'instance-0000001d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000001d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:42.392 4076 3051054567848 MainThread ERROR nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Failed to change vm state of instance-0000001d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. 2021-09-10 16:53:42.392 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Got request to destroy instance 2021-09-10 16:53:42.428 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-82484de0-b088-4384-978f-24300b1126a4 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:42.428 4076 3051054569080 MainThread DEBUG nova.virt.hyperv.vmops [req-82484de0-b088-4384-978f-24300b1126a4 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:42.485 4076 3051054569080 MainThread DEBUG nova.compute.manager [req-82484de0-b088-4384-978f-24300b1126a4 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:42.518 4076 3051054569080 MainThread INFO nova.compute.manager [req-82484de0-b088-4384-978f-24300b1126a4 - - - - -] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:42.998 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Successfully created port: 37fe3be0-a2b0-49a2-ae82-98cc905f3b1f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:53:44.334 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:53:44.334 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:44.334 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:44.350 4076 3051054567848 MainThread DEBUG os_win.utils.compute.vmutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Successfully changed vm state of instance-0000001d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:53:44.350 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Successfully changed state of VM instance-0000001d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:53:44.396 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Waiting for WMI job: {'InstanceID': '46D7DC28-F911-4B8D-B66F-B17F23A5538B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16394), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:44.537 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] WMI job succeeded: {'InstanceID': '46D7DC28-F911-4B8D-B66F-B17F23A5538B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32071), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:53:44.537 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:44.537 4076 3051054567848 MainThread DEBUG os_win.utils.pathutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Removing directory: C:\openstack\instances\instance-0000001d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:53:44.537 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:44.537 4076 3051054567848 MainThread ERROR nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Traceback (most recent call last): 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(*args, **kwargs) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] admin_password, network_info, block_device_info) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(self, *args, **kwds) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] should_plug_vifs=False) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'req_state': req_state}) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] time.sleep(sleep_time) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return f(*args, **kwargs) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return self._wait_for_job(job_path) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] error_desc=err_desc) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During handling of the above exception, another exception occurred: 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Traceback (most recent call last): 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] yield resources 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] accel_info=accel_info) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(*args, **kwargs) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] admin_password, network_info, block_device_info) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(self, *args, **kwds) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] should_plug_vifs=False) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'req_state': req_state}) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] time.sleep(sleep_time) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return f(*args, **kwargs) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return self._wait_for_job(job_path) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] error_desc=err_desc) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:44.537 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:44.553 4076 3051054567848 MainThread INFO nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Terminating instance 2021-09-10 16:53:44.553 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:53:44.553 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Got request to destroy instance 2021-09-10 16:53:44.568 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:53:44.568 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'target_lun': 0, 'volume_id': 'a0ac4351-dd95-40e6-bcc7-f7e316d40088', 'auth_method': 'CHAP', 'auth_username': 'WUVTEq8Na5sjUfeUS977', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:44.568 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:53:44.685 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:53:44.685 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 16:53:44.748 4076 3051054570136 MainThread DEBUG nova.virt.hyperv.volumeops [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936', 'target_lun': 0, 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'auth_method': 'CHAP', 'auth_username': 'cLGGMECT57NdrLs34ud4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '9107836a-481d-4cb5-9c7d-6432f4cd5309', 'attached_at': '', 'detached_at': '', 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'serial': '6fe38b5b-f580-4b49-8822-02f1519e6936'} to instance-0000001f _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:53:44.748 4076 3051054570136 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936', 'target_lun': 0, 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'auth_method': 'CHAP', 'auth_username': 'cLGGMECT57NdrLs34ud4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:44.748 4076 3051054570136 MainThread INFO os_brick.initiator.windows.iscsi [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:53:44.784 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-a0ac4351-dd95-40e6-bcc7-f7e316d40088 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 16:53:44.784 4076 3051054570136 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:53:44.784 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] <== disconnect_volume: return (215ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:44.784 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:44.784 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "instance-0000001d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:44.784 4076 3051054567848 MainThread INFO nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Took 0.23 seconds to destroy the instance on the hypervisor. 2021-09-10 16:53:45.297 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:45.381 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:53:45.529 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:45.529 4076 3051054571192 MainThread DEBUG nova.virt.hardware [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:53:45.529 4076 3051054571192 MainThread INFO nova.compute.claims [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Claim successful on node n-h1-808474-1 2021-09-10 16:53:45.654 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Successfully updated port: 37fe3be0-a2b0-49a2-ae82-98cc905f3b1f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:53:45.688 4076 3051054571016 GreenThread-19 DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Acquired lock "refresh_cache-ecbb6387-a846-41fc-9117-6880e641e5bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:45.688 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:53:45.721 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:45.750 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:45.836 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:53:45.936 4076 3051054568200 MainThread DEBUG nova.compute.manager [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Received event network-changed-37fe3be0-a2b0-49a2-ae82-98cc905f3b1f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:45.936 4076 3051054568200 MainThread DEBUG nova.compute.manager [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Refreshing instance network info cache due to event network-changed-37fe3be0-a2b0-49a2-ae82-98cc905f3b1f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:53:45.972 4076 3051054571192 MainThread DEBUG nova.compute.provider_tree [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:45.987 4076 3051054571192 MainThread DEBUG nova.scheduler.client.report [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:45.987 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:45.987 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:53:46.165 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:53:46.165 4076 3051092582472 GreenThread-20 DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:53:46.165 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:53:46.264 4076 3051054571192 MainThread INFO nova.virt.block_device [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Booting with volume f5252e88-87bd-471e-b572-a8d70f3c3790 at /dev/sda 2021-09-10 16:53:46.359 4076 3051092582472 GreenThread-20 DEBUG nova.policy [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '4826ecf435ad43b6984adee7a94b6ebc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7199a32f43ef47a1940566d570e22783', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:53:46.409 4076 3051054571192 MainThread DEBUG os_brick.utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:46.409 4076 3051054571192 MainThread DEBUG os_brick.utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:46.409 4076 3051054571192 MainThread DEBUG nova.virt.block_device [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Updating existing volume attachment record: 4c7caf97-8cde-4a77-8565-864296fffc74 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:53:46.561 4076 3051054571016 GreenThread-19 DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Updating instance_info_cache with network_info: [{"id": "37fe3be0-a2b0-49a2-ae82-98cc905f3b1f", "address": "fa:16:3e:7f:e3:e7", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37fe3be0-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:46.584 4076 3051054571016 GreenThread-19 DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Releasing lock "refresh_cache-ecbb6387-a846-41fc-9117-6880e641e5bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:46.584 4076 3051054571016 GreenThread-19 DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Instance network_info: |[{"id": "37fe3be0-a2b0-49a2-ae82-98cc905f3b1f", "address": "fa:16:3e:7f:e3:e7", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37fe3be0-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:53:46.584 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ecbb6387-a846-41fc-9117-6880e641e5bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:46.584 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Refreshing network info cache for port 37fe3be0-a2b0-49a2-ae82-98cc905f3b1f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:53:46.584 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Preparing to wait for external event network-vif-plugged-37fe3be0-a2b0-49a2-ae82-98cc905f3b1f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:53:46.584 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:46.584 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:46.584 4076 3051054569256 MainThread DEBUG os_win.utils.compute.vmutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating VM instance-00000021 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:53:47.360 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Successfully created port: b921f911-8db1-4922-bd91-9f5c2c47d1ec _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:53:47.486 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:53:48.753 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Successfully updated port: b921f911-8db1-4922-bd91-9f5c2c47d1ec _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:53:48.779 4076 3051092582472 GreenThread-20 DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Acquired lock "refresh_cache-f3fbc33e-18d4-47bf-b643-94d2edfcae5f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:48.779 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:53:48.939 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:53:49.114 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Updated VIF entry in instance network info cache for port 37fe3be0-a2b0-49a2-ae82-98cc905f3b1f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:53:49.114 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Updating instance_info_cache with network_info: [{"id": "37fe3be0-a2b0-49a2-ae82-98cc905f3b1f", "address": "fa:16:3e:7f:e3:e7", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37fe3be0-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:49.126 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [req-616bafcb-defc-49ff-8bbf-2181ddc6bfb9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ecbb6387-a846-41fc-9117-6880e641e5bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:49.792 4076 3051054568200 MainThread DEBUG nova.compute.manager [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Received event network-changed-b921f911-8db1-4922-bd91-9f5c2c47d1ec external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:49.792 4076 3051054568200 MainThread DEBUG nova.compute.manager [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Refreshing instance network info cache due to event network-changed-b921f911-8db1-4922-bd91-9f5c2c47d1ec. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:53:49.794 4076 3051054570136 MainThread DEBUG os_win.utils.storage.diskutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:53:49.889 4076 3051054570136 MainThread DEBUG os_win.utils.storage.diskutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:53:49.922 4076 3051054570136 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== connect_volume: return (5173ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '3600140539E1019FB39C4B4D82780189D'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:53:49.989 4076 3051092582472 GreenThread-20 DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Updating instance_info_cache with network_info: [{"id": "b921f911-8db1-4922-bd91-9f5c2c47d1ec", "address": "fa:16:3e:7a:67:db", "network": {"id": "1e7a7677-ebf7-4159-b113-9f9b3431daf6", "bridge": null, "label": "tempest-ServersTestBootFromVolume-771384563-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "7199a32f43ef47a1940566d570e22783", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb921f911-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:50.015 4076 3051092582472 GreenThread-20 DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Releasing lock "refresh_cache-f3fbc33e-18d4-47bf-b643-94d2edfcae5f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:50.015 4076 3051092582472 GreenThread-20 DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Instance network_info: |[{"id": "b921f911-8db1-4922-bd91-9f5c2c47d1ec", "address": "fa:16:3e:7a:67:db", "network": {"id": "1e7a7677-ebf7-4159-b113-9f9b3431daf6", "bridge": null, "label": "tempest-ServersTestBootFromVolume-771384563-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "7199a32f43ef47a1940566d570e22783", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb921f911-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:53:50.015 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f3fbc33e-18d4-47bf-b643-94d2edfcae5f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:50.030 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Refreshing network info cache for port b921f911-8db1-4922-bd91-9f5c2c47d1ec _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:53:50.412 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:50.451 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:53:50.596 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-9f18318e-b447-48e0-93b1-1c6bed20e989 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Received event network-vif-unplugged-76db5559-28e4-4b26-bfce-714d72b14bfd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:50.596 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-9f18318e-b447-48e0-93b1-1c6bed20e989 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:50.596 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-9f18318e-b447-48e0-93b1-1c6bed20e989 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:50.596 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-9f18318e-b447-48e0-93b1-1c6bed20e989 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] No waiting events found dispatching network-vif-unplugged-76db5559-28e4-4b26-bfce-714d72b14bfd pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:53:50.596 4076 3051092583000 MainThread WARNING nova.compute.manager [req-9f18318e-b447-48e0-93b1-1c6bed20e989 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Received unexpected event network-vif-unplugged-76db5559-28e4-4b26-bfce-714d72b14bfd for instance with vm_state building and task_state spawning. 2021-09-10 16:53:50.654 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:50.654 4076 3051092582648 MainThread DEBUG nova.virt.hardware [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:53:50.654 4076 3051092582648 MainThread INFO nova.compute.claims [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Claim successful on node n-h1-808474-1 2021-09-10 16:53:50.860 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.hostops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:50.876 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.hostops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:51.108 4076 3051092582648 MainThread DEBUG nova.compute.provider_tree [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:51.138 4076 3051092582648 MainThread DEBUG nova.scheduler.client.report [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:51.138 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:51.138 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:53:51.294 4076 3051092582648 MainThread DEBUG nova.compute.utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:53:51.294 4076 3051092583000 GreenThread-21 DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:53:51.294 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:53:51.319 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:53:51.513 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Updated VIF entry in instance network info cache for port b921f911-8db1-4922-bd91-9f5c2c47d1ec. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:53:51.513 4076 3051054568200 MainThread DEBUG nova.network.neutron [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Updating instance_info_cache with network_info: [{"id": "b921f911-8db1-4922-bd91-9f5c2c47d1ec", "address": "fa:16:3e:7a:67:db", "network": {"id": "1e7a7677-ebf7-4159-b113-9f9b3431daf6", "bridge": null, "label": "tempest-ServersTestBootFromVolume-771384563-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "7199a32f43ef47a1940566d570e22783", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb921f911-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:51.544 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [req-080c3a1d-f76a-4dc8-8191-64cc848ad390 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f3fbc33e-18d4-47bf-b643-94d2edfcae5f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:51.613 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:53:51.613 4076 3051092582648 MainThread INFO nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Spawning new instance 2021-09-10 16:53:51.629 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:51.629 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:51.629 4076 3051092582648 MainThread DEBUG nova.block_device [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:53:51.629 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:51.629 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:51.656 4076 3051092582648 MainThread DEBUG os_win.utils.pathutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Creating directory: C:\openstack\instances\instance-00000022 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:53:51.656 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000022\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:53:51.685 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Resizing VHD C:\openstack\instances\instance-00000022\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:53:51.749 4076 3051092583000 GreenThread-21 DEBUG nova.policy [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '92577cd3faa34912ba4767d0c2bde084', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '662be160c9d04068bc4711ac9c9433d7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:53:53.123 4076 3051054567848 MainThread INFO nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Took 8.34 seconds to detach 1 volumes for instance. 2021-09-10 16:53:53.123 4076 3051054567848 MainThread DEBUG nova.compute.claims [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:53:53.123 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:53.294 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:53:53.310 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:53:53.368 4076 3051054570136 MainThread DEBUG nova.objects.instance [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lazy-loading 'flavor' on Instance uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:53:53.425 4076 3051054570136 MainThread DEBUG oslo_concurrency.lockutils [req-1e3ffad4-1491-4936-a561-ba8a51b74cd8 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager.attach_volume..do_attach_volume" :: held 22.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:53.485 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Successfully created port: 73746707-7c0f-494e-b5b2-c9c883f64863 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:53:53.533 4076 3051054567848 MainThread DEBUG nova.compute.provider_tree [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.scheduler.client.report [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:53.562 4076 3051054567848 MainThread ERROR nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Traceback (most recent call last): 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(*args, **kwargs) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] admin_password, network_info, block_device_info) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(self, *args, **kwds) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] should_plug_vifs=False) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'req_state': req_state}) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] time.sleep(sleep_time) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return f(*args, **kwargs) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return self._wait_for_job(job_path) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] error_desc=err_desc) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] During handling of the above exception, another exception occurred: 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Traceback (most recent call last): 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] accel_info=accel_info) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(*args, **kwargs) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] admin_password, network_info, block_device_info) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return function(self, *args, **kwds) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] should_plug_vifs=False) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'req_state': req_state}) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] time.sleep(sleep_time) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self.force_reraise() 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] raise self.value 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return f(*args, **kwargs) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] return self._wait_for_job(job_path) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] error_desc=err_desc) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. 2021-09-10 16:53:53.562 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.compute.utils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Build of instance 079b5c5d-fe33-48d2-ba84-de08229901eb was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000001d' failed to start. Not enough memory in the system to start the virtual machine instance-0000001d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000001d' failed to start. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Not enough memory in the system to start the virtual machine instance-0000001d with ram size 1024 megabytes. (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) 'instance-0000001d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B930DB5B-44E5-40E4-8ED1-1572105A0C40) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:53:53.562 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:53:54.515 4076 3051092583528 MainThread DEBUG nova.compute.manager [req-ba69b7b3-9658-4881-958b-bd8c8fb8876c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Received event network-vif-plugged-37fe3be0-a2b0-49a2-ae82-98cc905f3b1f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:54.515 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [req-ba69b7b3-9658-4881-958b-bd8c8fb8876c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:54.515 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [req-ba69b7b3-9658-4881-958b-bd8c8fb8876c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:54.515 4076 3051092583528 MainThread DEBUG nova.compute.manager [req-ba69b7b3-9658-4881-958b-bd8c8fb8876c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Processing event network-vif-plugged-37fe3be0-a2b0-49a2-ae82-98cc905f3b1f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:53:54.548 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Using config drive for instance 2021-09-10 16:53:54.778 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Creating config drive at C:\openstack\instances\instance-00000021\configdrive.iso 2021-09-10 16:53:54.814 4076 3051054569256 MainThread DEBUG oslo_concurrency.processutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000021\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyrm3dc_4 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:53:54.989 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:55.000 4076 3051054567848 MainThread INFO nova.compute.manager [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 079b5c5d-fe33-48d2-ba84-de08229901eb] Took 1.44 seconds to deallocate network for instance. 2021-09-10 16:53:55.139 4076 3051054569256 MainThread DEBUG oslo_concurrency.processutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000021\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyrm3dc_4" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:53:55.298 4076 3051054567848 MainThread INFO nova.scheduler.client.report [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Deleted allocations for instance 079b5c5d-fe33-48d2-ba84-de08229901eb 2021-09-10 16:53:55.326 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-694051d5-68cf-4fa0-9b51-cd9e0a53c0a7 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "079b5c5d-fe33-48d2-ba84-de08229901eb" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 37.844s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:55.545 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Setting boot order for instance: instance-00000021: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:53:55.582 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:53:55.624 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'DDF0031F-ABF6-4130-8762-C44A1002FACE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14728), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:55.763 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:55.763 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:55.763 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:55.763 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:53:55.779 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DDF0031F-ABF6-4130-8762-C44A1002FACE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120355), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:55.796 4076 3051054569256 MainThread DEBUG os_win._utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.21422362327575684 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:55.796 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-1df61384-8315-4cc4-845a-869e45d0f6f7 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:55.816 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-1df61384-8315-4cc4-845a-869e45d0f6f7 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:55.871 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-1df61384-8315-4cc4-845a-869e45d0f6f7 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:55.889 4076 3051054567848 MainThread INFO nova.compute.manager [req-1df61384-8315-4cc4-845a-869e45d0f6f7 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:56.064 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Successfully updated port: 73746707-7c0f-494e-b5b2-c9c883f64863 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:53:56.092 4076 3051092583000 GreenThread-21 DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Acquired lock "refresh_cache-f0b03925-8953-422e-83b5-82f2f00745af" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:56.092 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:53:56.257 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:53:56.825 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'E4D0E69E-6BD4-4F8F-932A-C5A597B1618D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16268), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:56.982 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:56.982 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:56.982 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:56.982 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:53:56.982 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E4D0E69E-6BD4-4F8F-932A-C5A597B1618D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=135649), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:56.982 4076 3051054569256 MainThread DEBUG os_win._utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4003770351409912 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:57.031 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-cdb4e0d3-c33d-432a-a28c-9b97292a212e - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:57.031 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-cdb4e0d3-c33d-432a-a28c-9b97292a212e - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:57.083 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-cdb4e0d3-c33d-432a-a28c-9b97292a212e - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:57.127 4076 3051054567848 MainThread INFO nova.compute.manager [req-cdb4e0d3-c33d-432a-a28c-9b97292a212e - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:57.219 4076 3051092583000 GreenThread-21 DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Updating instance_info_cache with network_info: [{"id": "73746707-7c0f-494e-b5b2-c9c883f64863", "address": "fa:16:3e:fa:46:0e", "network": {"id": "04d8859a-8629-42fd-bdbd-a131a503ae71", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-344376700-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "662be160c9d04068bc4711ac9c9433d7", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap73746707-7c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:57.355 4076 3051092583000 GreenThread-21 DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Releasing lock "refresh_cache-f0b03925-8953-422e-83b5-82f2f00745af" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:57.355 4076 3051092583000 GreenThread-21 DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Instance network_info: |[{"id": "73746707-7c0f-494e-b5b2-c9c883f64863", "address": "fa:16:3e:fa:46:0e", "network": {"id": "04d8859a-8629-42fd-bdbd-a131a503ae71", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-344376700-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "662be160c9d04068bc4711ac9c9433d7", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap73746707-7c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:53:57.355 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Preparing to wait for external event network-vif-plugged-73746707-7c0f-494e-b5b2-c9c883f64863 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:53:57.355 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:57.355 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:57.355 4076 3051092582648 MainThread DEBUG os_win.utils.compute.vmutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Creating VM instance-00000022 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:53:57.373 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.livemigrationops [req-3c50aed4-453b-49ff-aed0-9ddeb0e8f00a ca2efda67fb64954b0bfece0cc597f88 398a3c674d0e41c1a7e4459b18602bb9 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] check_can_live_migrate_source called check_can_live_migrate_source c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:153 2021-09-10 16:53:57.373 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-3c50aed4-453b-49ff-aed0-9ddeb0e8f00a ca2efda67fb64954b0bfece0cc597f88 398a3c674d0e41c1a7e4459b18602bb9 - default default] source check data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=,old_vol_attachment_ids=,vifs=,wait_for_vif_plugged=) check_can_live_migrate_source c:\openstack\build\nova\nova\compute\manager.py:8151 2021-09-10 16:53:57.795 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Received event network-changed-73746707-7c0f-494e-b5b2-c9c883f64863 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:53:57.795 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Refreshing instance network info cache due to event network-changed-73746707-7c0f-494e-b5b2-c9c883f64863. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:53:57.795 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f0b03925-8953-422e-83b5-82f2f00745af" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:53:57.795 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Refreshing network info cache for port 73746707-7c0f-494e-b5b2-c9c883f64863 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:53:58.014 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'B7868DCD-427D-42FA-A968-D83DB1738122', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20898), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:58.158 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:58.158 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:58.158 4076 3051092583704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:58.172 4076 3051092583704 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:53:58.172 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B7868DCD-427D-42FA-A968-D83DB1738122', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122129), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:58.172 4076 3051054569256 MainThread DEBUG os_win._utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.590085744857788 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:58.206 4076 3051092583704 MainThread DEBUG nova.compute.manager [req-e03f3fd2-8caf-494a-8388-6e776ea8f494 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:58.206 4076 3051092583704 MainThread DEBUG nova.virt.hyperv.vmops [req-e03f3fd2-8caf-494a-8388-6e776ea8f494 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:58.273 4076 3051092583704 MainThread DEBUG nova.compute.manager [req-e03f3fd2-8caf-494a-8388-6e776ea8f494 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:58.298 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:53:58.335 4076 3051092583704 MainThread INFO nova.compute.manager [req-e03f3fd2-8caf-494a-8388-6e776ea8f494 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:59.222 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'DEC337EC-171B-4B3D-8AFD-EAF7780ADE54', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15474), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:53:59.281 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Updated VIF entry in instance network info cache for port 73746707-7c0f-494e-b5b2-c9c883f64863. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:53:59.281 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Updating instance_info_cache with network_info: [{"id": "73746707-7c0f-494e-b5b2-c9c883f64863", "address": "fa:16:3e:fa:46:0e", "network": {"id": "04d8859a-8629-42fd-bdbd-a131a503ae71", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-344376700-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "662be160c9d04068bc4711ac9c9433d7", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap73746707-7c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:53:59.294 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-8f2c0ff1-9513-4277-a19d-1dbeae916ff8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f0b03925-8953-422e-83b5-82f2f00745af" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:53:59.358 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:53:59.358 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:53:59.358 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:59.358 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:59.358 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DEC337EC-171B-4B3D-8AFD-EAF7780ADE54', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116950), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 53, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 53, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:53:59.358 4076 3051054569256 MainThread DEBUG os_win._utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7766573429107666 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:53:59.393 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-5f50b9d6-9a75-416e-af68-a88b7f7fd598 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:53:59.393 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-5f50b9d6-9a75-416e-af68-a88b7f7fd598 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:53:59.435 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-5f50b9d6-9a75-416e-af68-a88b7f7fd598 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:53:59.490 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:53:59.498 4076 3051054567848 MainThread INFO nova.compute.manager [req-5f50b9d6-9a75-416e-af68-a88b7f7fd598 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:53:59.498 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Spawning new instance 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG nova.block_device [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Preparing to wait for external event network-vif-plugged-b921f911-8db1-4922-bd91-9f5c2c47d1ec prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:53:59.498 4076 3051054571192 MainThread DEBUG os_win.utils.compute.vmutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Creating VM instance-00000020 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:53:59.984 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.volumeops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790', 'target_lun': 0, 'volume_id': 'f5252e88-87bd-471e-b572-a8d70f3c3790', 'auth_method': 'CHAP', 'auth_username': 'nVSjUhEE86QfhZaDKiyx', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'f3fbc33e-18d4-47bf-b643-94d2edfcae5f', 'attached_at': '', 'detached_at': '', 'volume_id': 'f5252e88-87bd-471e-b572-a8d70f3c3790', 'serial': 'f5252e88-87bd-471e-b572-a8d70f3c3790'} to instance-00000020 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:53:59.984 4076 3051054571192 MainThread DEBUG os_brick.initiator.windows.iscsi [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790', 'target_lun': 0, 'volume_id': 'f5252e88-87bd-471e-b572-a8d70f3c3790', 'auth_method': 'CHAP', 'auth_username': 'nVSjUhEE86QfhZaDKiyx', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:53:59.984 4076 3051054571192 MainThread INFO os_brick.initiator.windows.iscsi [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:53:59.984 4076 3051054571192 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:54:00.401 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '633A1E26-E004-4439-B3D2-3E3083451366', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18970), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:00.549 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:00.549 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:54:00.549 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:00.549 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:00.549 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '633A1E26-E004-4439-B3D2-3E3083451366', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=123002), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:00.549 4076 3051054569256 MainThread DEBUG os_win._utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.967005729675293 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:00.583 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-de9804ff-857e-4dd3-9bce-56bee9f8f479 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:00.583 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-de9804ff-857e-4dd3-9bce-56bee9f8f479 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:00.632 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-de9804ff-857e-4dd3-9bce-56bee9f8f479 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:00.660 4076 3051054567848 MainThread INFO nova.compute.manager [req-de9804ff-857e-4dd3-9bce-56bee9f8f479 - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:01.317 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:01.344 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:54:01.516 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:01.516 4076 3051054571368 MainThread DEBUG nova.virt.hardware [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:54:01.516 4076 3051054571368 MainThread INFO nova.compute.claims [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Claim successful on node n-h1-808474-1 2021-09-10 16:54:01.615 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '1E0FB862-CEDC-47C9-9EE6-0AD2B4F8BA5B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15279), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:01.672 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.hostops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:01.701 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.hostops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:01.749 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:01.763 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] VM Stopped (Lifecycle Event) 2021-09-10 16:54:01.763 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:01.763 4076 3051054568200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:01.763 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000021\' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000021\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)instance-000000215672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1E0FB862-CEDC-47C9-9EE6-0AD2B4F8BA5B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113374), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\nNot enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)\n\n'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C)", 'ErrorSummaryDescription': "'instance-00000021' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000021.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:01.763 4076 3051054569256 MainThread ERROR nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Failed to change vm state of instance-00000021 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. 2021-09-10 16:54:01.763 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Got request to destroy instance 2021-09-10 16:54:01.797 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-3e5efadd-2d44-49f5-9ddf-510ef2d4695d - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:01.797 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-3e5efadd-2d44-49f5-9ddf-510ef2d4695d - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:01.841 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-3e5efadd-2d44-49f5-9ddf-510ef2d4695d - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:01.889 4076 3051054567848 MainThread INFO nova.compute.manager [req-3e5efadd-2d44-49f5-9ddf-510ef2d4695d - - - - -] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:01.955 4076 3051054571368 MainThread DEBUG nova.compute.provider_tree [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:01.984 4076 3051054571368 MainThread DEBUG nova.scheduler.client.report [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:01.984 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:01.984 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:54:02.098 4076 3051054571368 MainThread DEBUG nova.compute.utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:54:02.098 4076 3051054567848 GreenThread-22 DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:54:02.098 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:54:02.152 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:54:02.363 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:54:02.363 4076 3051054571368 MainThread INFO nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Spawning new instance 2021-09-10 16:54:02.382 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:02.382 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:02.382 4076 3051054571368 MainThread DEBUG nova.block_device [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:54:02.382 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:02.382 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:02.399 4076 3051054571368 MainThread DEBUG os_win.utils.pathutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Creating directory: C:\openstack\instances\instance-00000023 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:54:02.399 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000023\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:54:02.424 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Resizing VHD C:\openstack\instances\instance-00000023\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:54:02.441 4076 3051054567848 GreenThread-22 DEBUG nova.policy [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '14929f631b514b1ba7de48ea6f7875b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12d8e551e4d4d599a26e3249f7560f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:54:02.607 4076 3051092583528 MainThread DEBUG nova.compute.manager [req-90de2360-05a4-44f9-a610-d827ca968d0e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Received event network-vif-plugged-73746707-7c0f-494e-b5b2-c9c883f64863 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:02.619 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [req-90de2360-05a4-44f9-a610-d827ca968d0e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:02.619 4076 3051092583528 MainThread DEBUG oslo_concurrency.lockutils [req-90de2360-05a4-44f9-a610-d827ca968d0e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:02.619 4076 3051092583528 MainThread DEBUG nova.compute.manager [req-90de2360-05a4-44f9-a610-d827ca968d0e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Processing event network-vif-plugged-73746707-7c0f-494e-b5b2-c9c883f64863 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:54:02.656 4076 3051092582648 MainThread INFO nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Using config drive for instance 2021-09-10 16:54:02.842 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:02.842 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:02.842 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:02.842 4076 3051092583880 MainThread ERROR nova.compute.manager [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Traceback (most recent call last): 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] self._delete_instance(context, instance, bdms) 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] NameError: name 'instant' is not defined 2021-09-10 16:54:02.842 4076 3051092583880 MainThread TRACE nova.compute.manager [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] 2021-09-10 16:54:02.905 4076 3051092582648 MainThread INFO nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Creating config drive at C:\openstack\instances\instance-00000022\configdrive.iso 2021-09-10 16:54:02.956 4076 3051092582648 MainThread DEBUG oslo_concurrency.processutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000022\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3kb8v_rx execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:54:03.114 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:03.194 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:03.249 4076 3051092583880 MainThread DEBUG nova.virt.hyperv.hostops [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:03.280 4076 3051092583880 MainThread DEBUG nova.virt.hyperv.hostops [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:03.296 4076 3051092582648 MainThread DEBUG oslo_concurrency.processutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000022\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3kb8v_rx" returned: 0 in 0.343s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:54:03.529 4076 3051092583880 MainThread DEBUG nova.compute.provider_tree [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:03.549 4076 3051092583880 MainThread DEBUG nova.scheduler.client.report [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:03.549 4076 3051092583880 MainThread DEBUG oslo_concurrency.lockutils [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:03.560 4076 3051092583880 MainThread INFO nova.compute.manager [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:54:03.560 4076 3051092583880 MainThread ERROR oslo_messaging.rpc.server [req-4bcae3a3-a4a0-461f-a480-57eddeef7a3d fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:54:03.560 4076 3051092583880 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:54:03.647 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Successfully created port: 12e6f6e2-aa56-4d3c-95f8-c5e715703f47 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:03.812 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Setting boot order for instance: instance-00000022: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:54:03.875 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:54:03.907 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': 'BD0F88EB-D09F-438A-9F8C-394556E0332E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16679), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:04.071 4076 3051092583880 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:04.071 4076 3051092583880 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:04.071 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:04.076 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:04.076 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BD0F88EB-D09F-438A-9F8C-394556E0332E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=125688), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:04.076 4076 3051092582648 MainThread DEBUG os_win._utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.20117449760437012 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:04.076 4076 3051092583880 MainThread DEBUG nova.compute.manager [req-1ccee9ca-3554-403f-80c1-ddb5790a23c3 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:04.076 4076 3051092583880 MainThread DEBUG nova.virt.hyperv.vmops [req-1ccee9ca-3554-403f-80c1-ddb5790a23c3 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:04.139 4076 3051092583880 MainThread DEBUG nova.compute.manager [req-1ccee9ca-3554-403f-80c1-ddb5790a23c3 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:04.179 4076 3051092583880 MainThread INFO nova.compute.manager [req-1ccee9ca-3554-403f-80c1-ddb5790a23c3 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:04.208 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:04.208 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:04.208 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:04.229 4076 3051054569256 MainThread DEBUG os_win.utils.compute.vmutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed vm state of instance-00000021 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:04.229 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed state of VM instance-00000021 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:04.280 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '4A6203A7-A08F-4D34-83E1-DE5E412D71D8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14006), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:04.388 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job succeeded: {'InstanceID': '4A6203A7-A08F-4D34-83E1-DE5E412D71D8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=37950), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:04.388 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:04.388 4076 3051054569256 MainThread DEBUG os_win.utils.pathutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Removing directory: C:\openstack\instances\instance-00000021 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:04.388 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:04.388 4076 3051054569256 MainThread ERROR nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Traceback (most recent call last): 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(*args, **kwargs) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] admin_password, network_info, block_device_info) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(self, *args, **kwds) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] should_plug_vifs=False) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'req_state': req_state}) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] time.sleep(sleep_time) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return f(*args, **kwargs) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return self._wait_for_job(job_path) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] error_desc=err_desc) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During handling of the above exception, another exception occurred: 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Traceback (most recent call last): 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] yield resources 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] accel_info=accel_info) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(*args, **kwargs) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] admin_password, network_info, block_device_info) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(self, *args, **kwds) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] should_plug_vifs=False) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'req_state': req_state}) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] time.sleep(sleep_time) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return f(*args, **kwargs) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return self._wait_for_job(job_path) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] error_desc=err_desc) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.388 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.388 4076 3051054569256 MainThread INFO nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Terminating instance 2021-09-10 16:54:04.410 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:04.410 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Got request to destroy instance 2021-09-10 16:54:04.410 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:04.410 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:04.430 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000021" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:04.430 4076 3051054569256 MainThread INFO nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:04.430 4076 3051054569256 MainThread DEBUG nova.compute.claims [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:04.430 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:04.546 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:04.563 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:04.661 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Successfully updated port: 12e6f6e2-aa56-4d3c-95f8-c5e715703f47 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:04.670 4076 3051054567848 GreenThread-22 DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Acquired lock "refresh_cache-c466fddc-e13c-4586-a808-4e92dcac06cc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:04.670 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:54:04.751 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:04.784 4076 3051054569256 MainThread DEBUG nova.compute.provider_tree [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:04.796 4076 3051054569256 MainThread DEBUG nova.scheduler.client.report [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:04.796 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:04.796 4076 3051054569256 MainThread ERROR nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Traceback (most recent call last): 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(*args, **kwargs) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] admin_password, network_info, block_device_info) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(self, *args, **kwds) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] should_plug_vifs=False) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'req_state': req_state}) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] time.sleep(sleep_time) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return f(*args, **kwargs) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return self._wait_for_job(job_path) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] error_desc=err_desc) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] During handling of the above exception, another exception occurred: 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Traceback (most recent call last): 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] accel_info=accel_info) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(*args, **kwargs) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] admin_password, network_info, block_device_info) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return function(self, *args, **kwds) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] should_plug_vifs=False) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'req_state': req_state}) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] time.sleep(sleep_time) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self.force_reraise() 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] raise self.value 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return f(*args, **kwargs) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] return self._wait_for_job(job_path) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] error_desc=err_desc) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. 2021-09-10 16:54:04.796 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.compute.utils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Build of instance ecbb6387-a846-41fc-9117-6880e641e5bd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000021' failed to start. Not enough memory in the system to start the virtual machine instance-00000021. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000021' failed to start. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Not enough memory in the system to start the virtual machine instance-00000021 with ram size 1024 megabytes. (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) 'instance-00000021' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5672F6EF-E396-4EBA-9CCE-9C98CCF5FA7C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:04.815 4076 3051054569256 MainThread DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:54:04.921 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Received event network-changed-12e6f6e2-aa56-4d3c-95f8-c5e715703f47 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:04.921 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Refreshing instance network info cache due to event network-changed-12e6f6e2-aa56-4d3c-95f8-c5e715703f47. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:05.029 4076 3051054571192 MainThread DEBUG os_win.utils.storage.diskutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:54:05.117 4076 3051054571192 MainThread DEBUG os_win.utils.storage.diskutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:54:05.117 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': 'E6F78C15-C165-4328-99D3-00C26E3F6E32', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21397), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:05.156 4076 3051054571192 MainThread DEBUG os_brick.initiator.windows.iscsi [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] <== connect_volume: return (5171ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive2', 'number': 2, 'scsi_wwn': '36001405BE2D2BDFD62C431C8D40CCDA1'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:54:05.251 4076 3051092585288 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:05.251 4076 3051092585288 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:05.251 4076 3051092585112 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:05.264 4076 3051092585112 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:05.281 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E6F78C15-C165-4328-99D3-00C26E3F6E32', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119794), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:05.281 4076 3051092582648 MainThread DEBUG os_win._utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4067466259002686 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:05.281 4076 3051092585112 MainThread DEBUG nova.compute.manager [req-0842b0db-5777-457f-8cb4-b4efc1b32295 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:05.281 4076 3051092585112 MainThread DEBUG nova.virt.hyperv.vmops [req-0842b0db-5777-457f-8cb4-b4efc1b32295 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:05.330 4076 3051092585112 MainThread DEBUG nova.compute.manager [req-0842b0db-5777-457f-8cb4-b4efc1b32295 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:05.372 4076 3051092585112 MainThread INFO nova.compute.manager [req-0842b0db-5777-457f-8cb4-b4efc1b32295 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:05.392 4076 3051054569256 MainThread DEBUG nova.network.neutron [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:05.408 4076 3051054569256 MainThread INFO nova.compute.manager [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: ecbb6387-a846-41fc-9117-6880e641e5bd] Took 0.59 seconds to deallocate network for instance. 2021-09-10 16:54:05.576 4076 3051054569256 MainThread INFO nova.scheduler.client.report [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Deleted allocations for instance ecbb6387-a846-41fc-9117-6880e641e5bd 2021-09-10 16:54:05.608 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-cf04c780-a4c1-49ad-8018-7fe6422c33e6 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "ecbb6387-a846-41fc-9117-6880e641e5bd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:05.657 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:05.657 4076 3051054567848 GreenThread-22 DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Updating instance_info_cache with network_info: [{"id": "12e6f6e2-aa56-4d3c-95f8-c5e715703f47", "address": "fa:16:3e:09:e1:f9", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap12e6f6e2-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:05.701 4076 3051054567848 GreenThread-22 DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Releasing lock "refresh_cache-c466fddc-e13c-4586-a808-4e92dcac06cc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:05.701 4076 3051054567848 GreenThread-22 DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Instance network_info: |[{"id": "12e6f6e2-aa56-4d3c-95f8-c5e715703f47", "address": "fa:16:3e:09:e1:f9", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap12e6f6e2-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:54:05.701 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-c466fddc-e13c-4586-a808-4e92dcac06cc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:05.701 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Refreshing network info cache for port 12e6f6e2-aa56-4d3c-95f8-c5e715703f47 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:05.701 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Preparing to wait for external event network-vif-plugged-12e6f6e2-aa56-4d3c-95f8-c5e715703f47 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:05.701 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:05.701 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:05.701 4076 3051054571368 MainThread DEBUG os_win.utils.compute.vmutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Creating VM instance-00000023 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:54:05.788 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:05.788 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:05.788 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:05.788 4076 3051092583352 MainThread ERROR nova.compute.manager [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Traceback (most recent call last): 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] self._delete_instance(context, instance, bdms) 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] NameError: name 'instant' is not defined 2021-09-10 16:54:05.788 4076 3051092583352 MainThread TRACE nova.compute.manager [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] 2021-09-10 16:54:05.857 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.063s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:05.955 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:05.985 4076 3051092583352 MainThread DEBUG nova.virt.hyperv.hostops [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:06.010 4076 3051092583352 MainThread DEBUG nova.virt.hyperv.hostops [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:06.242 4076 3051092583352 MainThread DEBUG nova.compute.provider_tree [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:06.263 4076 3051092583352 MainThread DEBUG nova.scheduler.client.report [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:06.265 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:06.265 4076 3051092583352 MainThread INFO nova.compute.manager [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:54:06.265 4076 3051092583352 MainThread ERROR oslo_messaging.rpc.server [req-6256ff2c-7708-4f63-ba78-bc95f60c66e8 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:54:06.265 4076 3051092583352 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:54:06.310 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': 'C0BE3217-2574-4781-9591-04E8B0281689', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14936), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:06.445 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:06.445 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:06.445 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:06.445 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:06.460 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C0BE3217-2574-4781-9591-04E8B0281689', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115715), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:06.460 4076 3051092582648 MainThread DEBUG os_win._utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.585615873336792 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:06.472 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-f8bf300a-1627-41eb-9e75-e672c45a94bd - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:06.472 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-f8bf300a-1627-41eb-9e75-e672c45a94bd - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:06.487 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Updated VIF entry in instance network info cache for port 12e6f6e2-aa56-4d3c-95f8-c5e715703f47. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:06.497 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Updating instance_info_cache with network_info: [{"id": "12e6f6e2-aa56-4d3c-95f8-c5e715703f47", "address": "fa:16:3e:09:e1:f9", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap12e6f6e2-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:06.497 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-295f18be-1b08-4fbe-9594-f96bc01b8c21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-c466fddc-e13c-4586-a808-4e92dcac06cc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:06.522 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-f8bf300a-1627-41eb-9e75-e672c45a94bd - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:06.546 4076 3051054569256 MainThread INFO nova.compute.manager [req-f8bf300a-1627-41eb-9e75-e672c45a94bd - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:06.725 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:07.029 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:07.029 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:07.029 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:07.029 4076 3051092584760 MainThread ERROR nova.compute.manager [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Traceback (most recent call last): 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] self._delete_instance(context, instance, bdms) 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] NameError: name 'instant' is not defined 2021-09-10 16:54:07.029 4076 3051092584760 MainThread TRACE nova.compute.manager [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] 2021-09-10 16:54:07.099 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.062s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:07.203 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:07.233 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.hostops [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:07.254 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.hostops [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:07.485 4076 3051092584760 MainThread DEBUG nova.compute.provider_tree [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:07.507 4076 3051092584760 MainThread DEBUG nova.scheduler.client.report [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:07.507 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:07.507 4076 3051092584760 MainThread INFO nova.compute.manager [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:54:07.514 4076 3051092584760 MainThread ERROR oslo_messaging.rpc.server [req-cda79008-f8c5-4f97-832a-491c086360b8 68a189ea096f4b08ae59e252907de4ff be36af928b5149e58720b1f68dda425d - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:54:07.514 4076 3051092584760 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:54:07.514 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': 'EAAEE770-FB0B-441F-9130-27AA88C400E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17101), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:07.653 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:07.653 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:07.653 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:07.653 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:07.653 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EAAEE770-FB0B-441F-9130-27AA88C400E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115289), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:07.653 4076 3051092582648 MainThread DEBUG os_win._utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7785263061523438 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:07.663 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-248e0564-15aa-49ed-a970-d049ed909d09 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:07.663 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-248e0564-15aa-49ed-a970-d049ed909d09 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:07.717 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-248e0564-15aa-49ed-a970-d049ed909d09 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:07.748 4076 3051054567848 MainThread INFO nova.compute.manager [req-248e0564-15aa-49ed-a970-d049ed909d09 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:08.003 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.003 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.003 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:08.003 4076 3051054567848 MainThread ERROR nova.compute.manager [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Traceback (most recent call last): 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] self._delete_instance(context, instance, bdms) 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] NameError: name 'instant' is not defined 2021-09-10 16:54:08.003 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] 2021-09-10 16:54:08.062 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.062s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:08.173 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.216 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:08.236 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:08.467 4076 3051054567848 MainThread DEBUG nova.compute.provider_tree [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:08.486 4076 3051054567848 MainThread DEBUG nova.scheduler.client.report [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:08.486 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:08.486 4076 3051054567848 MainThread INFO nova.compute.manager [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:54:08.505 4076 3051054567848 MainThread ERROR oslo_messaging.rpc.server [req-4bffa5f4-a8d7-4207-ad1e-6da8553f2a22 fac7f2174628404cb8576928225b4394 fa8236f1477243dfafd0682b608fc659 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:54:08.505 4076 3051054567848 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:54:08.578 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.593 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:54:08.674 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': '0619DE57-F18F-466E-A501-5BD120394280', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14963), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:08.740 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.748 4076 3051054569256 MainThread DEBUG nova.virt.hardware [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:54:08.748 4076 3051054569256 MainThread INFO nova.compute.claims [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Claim successful on node n-h1-808474-1 2021-09-10 16:54:08.817 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0619DE57-F18F-466E-A501-5BD120394280', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=127813), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:08.817 4076 3051092582648 MainThread DEBUG os_win._utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9425506591796875 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:08.845 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:08.845 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:08.845 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:08.845 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:08.863 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-bf5445cf-cfb6-4583-857b-f8aea7bfb9d9 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:08.863 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-bf5445cf-cfb6-4583-857b-f8aea7bfb9d9 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:08.890 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:08.908 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:08.924 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-bf5445cf-cfb6-4583-857b-f8aea7bfb9d9 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:08.935 4076 3051054567848 MainThread INFO nova.compute.manager [req-bf5445cf-cfb6-4583-857b-f8aea7bfb9d9 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:09.138 4076 3051054569256 MainThread DEBUG nova.compute.provider_tree [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:09.138 4076 3051054569256 MainThread DEBUG nova.scheduler.client.report [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:09.138 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:09.138 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:54:09.216 4076 3051054569256 MainThread DEBUG nova.compute.utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:54:09.216 4076 3051054567848 GreenThread-23 DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:54:09.216 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:54:09.235 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:54:09.362 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:54:09.362 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Spawning new instance 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG nova.block_device [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG os_win.utils.pathutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating directory: C:\openstack\instances\instance-00000024 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:54:09.389 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000024\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:54:09.406 4076 3051054567848 GreenThread-23 DEBUG nova.policy [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd33d11497a91483d93fe0ed8f5c26603', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'efc5deaa89294889821f303976e2d776', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:54:09.436 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Resizing VHD C:\openstack\instances\instance-00000024\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:54:09.859 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': 'A1FA4F53-B755-44FA-AB16-A04AFB117212', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18094), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:10.014 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:10.014 4076 3051092583352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:10.014 4076 3051092584760 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:10.014 4076 3051092584760 MainThread INFO nova.compute.manager [-] [instance: f0b03925-8953-422e-83b5-82f2f00745af] VM Stopped (Lifecycle Event) 2021-09-10 16:54:10.014 4076 3051092582648 MainThread ERROR os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000022\' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000022\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)instance-000000229B6B55FD-1886-4DC3-B00B-54CC39568B6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A1FA4F53-B755-44FA-AB16-A04AFB117212', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=134079), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\nNot enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)\n\n'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B)", 'ErrorSummaryDescription': "'instance-00000022' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000022.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:10.014 4076 3051092582648 MainThread ERROR nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Failed to change vm state of instance-00000022 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. 2021-09-10 16:54:10.036 4076 3051092582648 MainThread INFO nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Got request to destroy instance 2021-09-10 16:54:10.046 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-52f4ca4e-c478-4a19-9e42-91a82a36f766 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:10.046 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.vmops [req-52f4ca4e-c478-4a19-9e42-91a82a36f766 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:10.094 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-52f4ca4e-c478-4a19-9e42-91a82a36f766 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:10.123 4076 3051092584760 MainThread INFO nova.compute.manager [req-52f4ca4e-c478-4a19-9e42-91a82a36f766 - - - - -] [instance: f0b03925-8953-422e-83b5-82f2f00745af] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:10.530 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Successfully created port: e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:11.563 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Successfully updated port: e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:11.563 4076 3051054567848 GreenThread-23 DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Acquired lock "refresh_cache-6f3c13e4-15bf-4dd6-ad7e-7870384f5220" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:11.563 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:54:11.719 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:12.007 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Received event network-changed-e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:12.007 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Refreshing instance network info cache due to event network-changed-e0a6eaf9-527b-44d8-8e4e-5beaa55bf373. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:12.048 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:12.048 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.048 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.082 4076 3051092582648 MainThread DEBUG os_win.utils.compute.vmutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Successfully changed vm state of instance-00000022 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:12.082 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Successfully changed state of VM instance-00000022 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:12.140 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Waiting for WMI job: {'InstanceID': '603F5167-52B8-4190-A123-91B75080E860', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16671), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:12.267 4076 3051092582648 MainThread DEBUG os_win.utils.jobutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] WMI job succeeded: {'InstanceID': '603F5167-52B8-4190-A123-91B75080E860', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=33518), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:12.267 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.267 4076 3051092582648 MainThread DEBUG os_win.utils.pathutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Removing directory: C:\openstack\instances\instance-00000022 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:12.267 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.267 4076 3051092582648 MainThread ERROR nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Traceback (most recent call last): 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(*args, **kwargs) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] admin_password, network_info, block_device_info) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(self, *args, **kwds) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] should_plug_vifs=False) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'req_state': req_state}) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] time.sleep(sleep_time) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return f(*args, **kwargs) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return self._wait_for_job(job_path) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] error_desc=err_desc) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] During handling of the above exception, another exception occurred: 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Traceback (most recent call last): 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] yield resources 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] accel_info=accel_info) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(*args, **kwargs) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] admin_password, network_info, block_device_info) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(self, *args, **kwds) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] should_plug_vifs=False) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'req_state': req_state}) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] time.sleep(sleep_time) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return f(*args, **kwargs) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return self._wait_for_job(job_path) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] error_desc=err_desc) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.267 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.267 4076 3051092582648 MainThread INFO nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Terminating instance 2021-09-10 16:54:12.267 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:12.267 4076 3051092582648 MainThread INFO nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Got request to destroy instance 2021-09-10 16:54:12.289 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:12.289 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.289 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "instance-00000022" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.305 4076 3051092582648 MainThread INFO nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:12.305 4076 3051092582648 MainThread DEBUG nova.compute.claims [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:12.305 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.305 4076 3051054567848 GreenThread-23 DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Updating instance_info_cache with network_info: [{"id": "e0a6eaf9-527b-44d8-8e4e-5beaa55bf373", "address": "fa:16:3e:84:2a:83", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape0a6eaf9-52", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:12.344 4076 3051054567848 GreenThread-23 DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Releasing lock "refresh_cache-6f3c13e4-15bf-4dd6-ad7e-7870384f5220" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:12.344 4076 3051054567848 GreenThread-23 DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Instance network_info: |[{"id": "e0a6eaf9-527b-44d8-8e4e-5beaa55bf373", "address": "fa:16:3e:84:2a:83", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape0a6eaf9-52", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:54:12.344 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6f3c13e4-15bf-4dd6-ad7e-7870384f5220" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:12.344 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Refreshing network info cache for port e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:12.344 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Preparing to wait for external event network-vif-plugged-e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:12.344 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.344 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.344 4076 3051054569256 MainThread DEBUG os_win.utils.compute.vmutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating VM instance-00000024 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:54:12.456 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.hostops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:12.467 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.hostops [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:12.530 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Took 14.42 seconds for pre_live_migration on destination host n-h2-808474-1. 2021-09-10 16:54:12.561 4076 3051092583000 MainThread DEBUG nova.compute.manager [-] live_migration data is HyperVLiveMigrateData(is_shared_instance_path=False,is_volume_backed=False,migration=Migration(79036b7d-5bae-483a-946d-f06145fd8c02),old_vol_attachment_ids={6fe38b5b-f580-4b49-8822-02f1519e6936='db68e464-635a-43fc-81e3-e6681a734234'},vifs=[VIFMigrateData],wait_for_vif_plugged=True) _do_live_migration c:\openstack\build\nova\nova\compute\manager.py:8499 2021-09-10 16:54:12.561 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.livemigrationops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] live_migration called live_migration c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py:52 2021-09-10 16:54:12.561 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.561 4076 3051092583000 MainThread INFO nova.virt.hyperv.serialconsoleops [-] Stopping instance instance-0000001f serial console handler. 2021-09-10 16:54:12.561 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000001f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.716 4076 3051092582648 MainThread DEBUG nova.compute.provider_tree [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:12.752 4076 3051092582648 MainThread DEBUG nova.scheduler.client.report [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:12.752 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.752 4076 3051092582648 MainThread ERROR nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Traceback (most recent call last): 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(*args, **kwargs) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] admin_password, network_info, block_device_info) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(self, *args, **kwds) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] should_plug_vifs=False) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'req_state': req_state}) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] time.sleep(sleep_time) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return f(*args, **kwargs) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return self._wait_for_job(job_path) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] error_desc=err_desc) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] During handling of the above exception, another exception occurred: 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Traceback (most recent call last): 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] accel_info=accel_info) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(*args, **kwargs) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] admin_password, network_info, block_device_info) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return function(self, *args, **kwds) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] should_plug_vifs=False) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'req_state': req_state}) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] time.sleep(sleep_time) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self.force_reraise() 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] raise self.value 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return f(*args, **kwargs) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] return self._wait_for_job(job_path) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] error_desc=err_desc) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. 2021-09-10 16:54:12.752 4076 3051092582648 MainThread TRACE nova.compute.manager [instance: f0b03925-8953-422e-83b5-82f2f00745af] 2021-09-10 16:54:12.752 4076 3051092582648 MainThread DEBUG nova.compute.utils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:12.752 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Build of instance f0b03925-8953-422e-83b5-82f2f00745af was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000022' failed to start. Not enough memory in the system to start the virtual machine instance-00000022. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000022' failed to start. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Not enough memory in the system to start the virtual machine instance-00000022 with ram size 1024 megabytes. (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) 'instance-00000022' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9B6B55FD-1886-4DC3-B00B-54CC39568B6B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:12.763 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:12.763 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:12.763 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:12.763 4076 3051092582648 MainThread DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:54:12.813 4076 3051092583000 MainThread DEBUG os_win.utils.compute.livemigrationutils [-] Getting live migration networks for host: n-h2-808474-1 _get_ip_address_list c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:172 2021-09-10 16:54:12.970 4076 3051092583000 MainThread DEBUG os_win.utils.compute.livemigrationutils [-] Starting live migration for VM: instance-0000001f _live_migrate_vm c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py:162 2021-09-10 16:54:12.997 4076 3051092585816 MainThread DEBUG nova.compute.manager [req-55013ec0-cddb-4718-a0cc-16c9bf885f78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Received event network-vif-plugged-b921f911-8db1-4922-bd91-9f5c2c47d1ec external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:12.997 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [req-55013ec0-cddb-4718-a0cc-16c9bf885f78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:12.997 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [req-55013ec0-cddb-4718-a0cc-16c9bf885f78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:12.997 4076 3051092585816 MainThread DEBUG nova.compute.manager [req-55013ec0-cddb-4718-a0cc-16c9bf885f78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Processing event network-vif-plugged-b921f911-8db1-4922-bd91-9f5c2c47d1ec _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:54:13.044 4076 3051092583000 MainThread DEBUG os_win.utils.jobutils [-] Waiting for WMI job: {'InstanceID': '2A85AF7E-6AD5-4715-AB50-1053BE29B18B', 'Description': 'Moving Virtual Machine and Storage', 'ElementName': 'Moving Virtual Machine and Storage', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21682), 'Cancellable': True, 'JobType': 306, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 1}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:13.044 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Using config drive for instance 2021-09-10 16:54:13.135 4076 3051092585816 MainThread DEBUG nova.compute.manager [req-1c44fe08-e590-4055-b829-d77a31fcb675 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-vif-unplugged-5beac292-790e-4830-8f59-e411516f08cc external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:13.135 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [req-1c44fe08-e590-4055-b829-d77a31fcb675 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:13.135 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [req-1c44fe08-e590-4055-b829-d77a31fcb675 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:13.135 4076 3051092585816 MainThread DEBUG nova.compute.manager [req-1c44fe08-e590-4055-b829-d77a31fcb675 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] No waiting events found dispatching network-vif-unplugged-5beac292-790e-4830-8f59-e411516f08cc pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:54:13.135 4076 3051092585816 MainThread DEBUG nova.compute.manager [req-1c44fe08-e590-4055-b829-d77a31fcb675 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-vif-unplugged-5beac292-790e-4830-8f59-e411516f08cc for instance with task_state migrating. _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10284 2021-09-10 16:54:13.252 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Creating config drive at C:\openstack\instances\instance-00000020\configdrive.iso 2021-09-10 16:54:13.311 4076 3051054571192 MainThread DEBUG oslo_concurrency.processutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000020\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu7lbmyy5 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:54:13.487 4076 3051092582648 MainThread DEBUG nova.network.neutron [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:13.514 4076 3051092582648 MainThread INFO nova.compute.manager [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] [instance: f0b03925-8953-422e-83b5-82f2f00745af] Took 0.75 seconds to deallocate network for instance. 2021-09-10 16:54:13.514 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:13.654 4076 3051054571192 MainThread DEBUG oslo_concurrency.processutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000020\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu7lbmyy5" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:54:13.747 4076 3051092582648 MainThread INFO nova.scheduler.client.report [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Deleted allocations for instance f0b03925-8953-422e-83b5-82f2f00745af 2021-09-10 16:54:13.780 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-55bdbb09-3a3e-4ad4-a4a6-a6ab41c6d191 92577cd3faa34912ba4767d0c2bde084 662be160c9d04068bc4711ac9c9433d7 - default default] Lock "f0b03925-8953-422e-83b5-82f2f00745af" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 23.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:13.819 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Updated VIF entry in instance network info cache for port e0a6eaf9-527b-44d8-8e4e-5beaa55bf373. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:13.819 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Updating instance_info_cache with network_info: [{"id": "e0a6eaf9-527b-44d8-8e4e-5beaa55bf373", "address": "fa:16:3e:84:2a:83", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape0a6eaf9-52", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:13.842 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-6d845247-a2e5-4450-9d71-83a72c4af8f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6f3c13e4-15bf-4dd6-ad7e-7870384f5220" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:14.101 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Setting boot order for instance: instance-00000020: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:54:14.140 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:54:14.176 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': '194E87B5-6724-4143-A1EE-10854A3C2EF9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15693), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:14.310 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:14.310 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] VM Stopped (Lifecycle Event) 2021-09-10 16:54:14.310 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:14.330 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:14.330 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '194E87B5-6724-4143-A1EE-10854A3C2EF9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114915), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:14.330 4076 3051054571192 MainThread DEBUG os_win._utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18971705436706543 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:14.330 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-af9d6eef-d475-4a3c-9797-e20a556f8763 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:14.330 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-af9d6eef-d475-4a3c-9797-e20a556f8763 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:14.393 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-af9d6eef-d475-4a3c-9797-e20a556f8763 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:14.421 4076 3051054567848 MainThread INFO nova.compute.manager [req-af9d6eef-d475-4a3c-9797-e20a556f8763 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:15.063 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-2060a28c-5883-4c27-9891-b202bc7ef1f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Received event network-vif-plugged-12e6f6e2-aa56-4d3c-95f8-c5e715703f47 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:15.063 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-2060a28c-5883-4c27-9891-b202bc7ef1f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:15.063 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-2060a28c-5883-4c27-9891-b202bc7ef1f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:15.063 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-2060a28c-5883-4c27-9891-b202bc7ef1f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Processing event network-vif-plugged-12e6f6e2-aa56-4d3c-95f8-c5e715703f47 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:54:15.110 4076 3051054571368 MainThread INFO nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Using config drive for instance 2021-09-10 16:54:15.272 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-changed-5beac292-790e-4830-8f59-e411516f08cc external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:15.272 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing instance network info cache due to event network-changed-5beac292-790e-4830-8f59-e411516f08cc. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:15.272 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:15.272 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing network info cache for port 5beac292-790e-4830-8f59-e411516f08cc _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:15.357 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': 'BB37A856-B5DE-4562-BC45-16C39FC5CE48', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16419), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:15.476 4076 3051054571368 MainThread INFO nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Creating config drive at C:\openstack\instances\instance-00000023\configdrive.iso 2021-09-10 16:54:15.517 4076 3051054571368 MainThread DEBUG oslo_concurrency.processutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000023\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3pced7io execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:54:15.687 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BB37A856-B5DE-4562-BC45-16C39FC5CE48', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115447), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:15.687 4076 3051054571192 MainThread DEBUG os_win._utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.5466344356536865 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:15.687 4076 3051092582648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:15.687 4076 3051092582648 MainThread INFO nova.compute.manager [-] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] VM Stopped (Lifecycle Event) 2021-09-10 16:54:15.687 4076 3051092585640 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:15.687 4076 3051092585640 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:15.721 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-053dda13-cc58-496e-8be5-e136448bf69e - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:15.721 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-053dda13-cc58-496e-8be5-e136448bf69e - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:15.766 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-053dda13-cc58-496e-8be5-e136448bf69e - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:15.810 4076 3051092582648 MainThread INFO nova.compute.manager [req-053dda13-cc58-496e-8be5-e136448bf69e - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:15.844 4076 3051054571368 MainThread DEBUG oslo_concurrency.processutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000023\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3pced7io" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:54:16.267 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Setting boot order for instance: instance-00000023: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:54:16.318 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:54:16.356 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '1CA8231F-4253-4744-AB6F-0DF4CC38D03F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15397), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:16.483 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:16.483 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:16.483 4076 3051092582472 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:16.483 4076 3051092582472 MainThread INFO nova.compute.manager [-] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] VM Stopped (Lifecycle Event) 2021-09-10 16:54:16.513 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1CA8231F-4253-4744-AB6F-0DF4CC38D03F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114538), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:16.513 4076 3051054571368 MainThread DEBUG os_win._utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1949474811553955 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:16.513 4076 3051092582472 MainThread DEBUG nova.compute.manager [req-d69923f8-c1ac-4a3b-b062-3f3a3bcba413 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:16.513 4076 3051092582472 MainThread DEBUG nova.virt.hyperv.vmops [req-d69923f8-c1ac-4a3b-b062-3f3a3bcba413 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:16.562 4076 3051092582472 MainThread DEBUG nova.compute.manager [req-d69923f8-c1ac-4a3b-b062-3f3a3bcba413 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:16.592 4076 3051092582472 MainThread INFO nova.compute.manager [req-d69923f8-c1ac-4a3b-b062-3f3a3bcba413 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:16.724 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': '6D639989-E102-4E22-98AF-0272CD055A11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19236), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:16.779 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated VIF entry in instance network info cache for port 5beac292-790e-4830-8f59-e411516f08cc. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:16.779 4076 3051092584760 MainThread DEBUG nova.network.neutron [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {"migrating_to": "n-h2-808474-1"}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:16.799 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [req-9098b029-d07f-4d59-8357-33cdc6339651 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:16.876 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:16.876 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] VM Stopped (Lifecycle Event) 2021-09-10 16:54:16.876 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:16.876 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:16.876 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6D639989-E102-4E22-98AF-0272CD055A11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=131234), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:16.876 4076 3051054571192 MainThread DEBUG os_win._utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.735924243927002 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:16.922 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-17026635-2654-4014-b748-3b14e20165fc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:16.922 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-17026635-2654-4014-b748-3b14e20165fc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:16.969 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-17026635-2654-4014-b748-3b14e20165fc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:16.997 4076 3051054567848 MainThread INFO nova.compute.manager [req-17026635-2654-4014-b748-3b14e20165fc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:17.552 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '15DC1736-0F7E-4556-BC72-E8C5846ACBC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15494), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:17.716 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '15DC1736-0F7E-4556-BC72-E8C5846ACBC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115390), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:17.716 4076 3051054571368 MainThread DEBUG os_win._utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3982505798339844 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:17.907 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': 'C920E212-4DF7-48E9-A68D-D001D8A89A1B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17010), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:18.156 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C920E212-4DF7-48E9-A68D-D001D8A89A1B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119813), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:18.156 4076 3051054571192 MainThread DEBUG os_win._utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.015657901763916 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:18.577 4076 3051092583000 MainThread ERROR os_win.utils.jobutils [-] WMI job failed: {'RawErrors': (('104Virtual machine migration operation for \'instance-0000001f\' failed at migration destination \'n-h2-808474-1\'. (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33)instance-0000001fFE3630CC-14E6-48A8-BC80-B2BC7FABCF33n-h2-808474-121026Microsoft-Windows-Hyper-V-VMMS50', '124\'instance-0000001f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33)instance-0000001fFE3630CC-14E6-48A8-BC80-B2BC7FABCF33%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2A85AF7E-6AD5-4715-AB50-1053BE29B18B', 'Description': 'Moving Virtual Machine and Storage', 'ElementName': 'Moving Virtual Machine and Storage', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(seconds=4, microseconds=570296), 'Cancellable': True, 'JobType': 306, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 20, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32768, 'ErrorDescription': "Virtual machine migration operation for 'instance-0000001f' failed at migration destination 'n-h2-808474-1'. (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33)\n\n'instance-0000001f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33)", 'ErrorSummaryDescription': 'Virtual machine migration operation failed at migration destination.\n\nCould not initialize memory: Ran out of memory (0x8007000E).'}. 2021-09-10 16:54:18.607 4076 3051092583000 MainThread ERROR nova.virt.hyperv.livemigrationops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Live migration failed. Attempting rollback.: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: Virtual machine migration operation failed at migration destination. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: Virtual machine migration operation for 'instance-0000001f' failed at migration destination 'n-h2-808474-1'. (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33) 'instance-0000001f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33) Error code: 32768. 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Traceback (most recent call last): 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\nova\nova\virt\hyperv\livemigrationops.py", line 78, in live_migration 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] migrate_disks=not shared_storage) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py", line 197, in live_migrate_vm 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] dest_host, migration_type) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\os-win\os_win\utils\compute\livemigrationutils.py", line 168, in _live_migrate_vm 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] self._jobutils.check_ret_val(ret_val, job_path) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] return self._wait_for_job(job_path) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] error_desc=err_desc) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: Virtual machine migration operation failed at migration destination. 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: Virtual machine migration operation for 'instance-0000001f' failed at migration destination 'n-h2-808474-1'. (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33) 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] 'instance-0000001f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FE3630CC-14E6-48A8-BC80-B2BC7FABCF33) Error code: 32768. 2021-09-10 16:54:18.607 4076 3051092583000 MainThread TRACE nova.virt.hyperv.livemigrationops [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] 2021-09-10 16:54:18.653 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Swapping old allocation on dict_keys(['6729133d-6e37-4062-bd58-46fc02d92e9d']) held by migration 79036b7d-5bae-483a-946d-f06145fd8c02 for instance 2021-09-10 16:54:18.685 4076 3051092583000 MainThread DEBUG nova.scheduler.client.report [-] Overwriting current allocation {'allocations': {'29611d7d-a4ec-4e5d-951c-2dc30bb9df24': {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}, 'generation': 33}}, 'project_id': '91237ebda1b54d2dbc761e48fdd93516', 'user_id': 'beffe9ba8529426fbb27e549254bd679', 'consumer_generation': 1} on consumer 9107836a-481d-4cb5-9c7d-6432f4cd5309 move_allocations c:\openstack\build\nova\nova\scheduler\client\report.py:1962 2021-09-10 16:54:18.747 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '81EC15FB-C86E-41B6-86A9-2F229F9AC2CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20277), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:18.897 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:18.897 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:18.897 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:18.897 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] VM Stopped (Lifecycle Event) 2021-09-10 16:54:18.906 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '81EC15FB-C86E-41B6-86A9-2F229F9AC2CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119369), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:18.906 4076 3051054571368 MainThread DEBUG os_win._utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.588134527206421 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:18.957 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-aaf80db6-bc22-41b4-a270-4bb76213bdb5 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:18.957 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-aaf80db6-bc22-41b4-a270-4bb76213bdb5 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:19.005 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-aaf80db6-bc22-41b4-a270-4bb76213bdb5 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:19.048 4076 3051054567848 MainThread INFO nova.compute.manager [req-aaf80db6-bc22-41b4-a270-4bb76213bdb5 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:19.181 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': 'F9C71AD8-58EF-4A4D-A4E3-8B3A83B18642', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15479), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:19.313 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:19.313 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:19.313 4076 3051092584760 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:19.313 4076 3051092584760 MainThread INFO nova.compute.manager [-] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] VM Stopped (Lifecycle Event) 2021-09-10 16:54:19.341 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9C71AD8-58EF-4A4D-A4E3-8B3A83B18642', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111735), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:19.341 4076 3051054571192 MainThread DEBUG os_win._utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.201127052307129 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:19.360 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-05dc1a09-0984-429a-8366-5d579a7459c5 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:19.379 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.vmops [req-05dc1a09-0984-429a-8366-5d579a7459c5 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:19.427 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-05dc1a09-0984-429a-8366-5d579a7459c5 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:19.463 4076 3051092584760 MainThread INFO nova.compute.manager [req-05dc1a09-0984-429a-8366-5d579a7459c5 - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:19.937 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '5151AC8C-C4EB-4EE3-93E0-26B797546909', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15212), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:20.079 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5151AC8C-C4EB-4EE3-93E0-26B797546909', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119144), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:20.079 4076 3051054571368 MainThread DEBUG os_win._utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7607181072235107 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:20.079 4076 3051092584760 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:20.079 4076 3051092584760 MainThread INFO nova.compute.manager [-] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] VM Stopped (Lifecycle Event) 2021-09-10 16:54:20.079 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:20.101 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:20.122 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-a66e0358-7e61-46a4-a1f8-950032ace513 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:20.122 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.vmops [req-a66e0358-7e61-46a4-a1f8-950032ace513 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:20.169 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-a66e0358-7e61-46a4-a1f8-950032ace513 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:20.200 4076 3051092584760 MainThread INFO nova.compute.manager [req-a66e0358-7e61-46a4-a1f8-950032ace513 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:20.389 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': 'B531EE69-26D0-43BC-8356-2C44368BF17A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15380), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:20.554 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000020\' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000020\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)instance-0000002030978065-CAA0-48E2-9DA4-C1446FB885A7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B531EE69-26D0-43BC-8356-2C44368BF17A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=138189), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\nNot enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)\n\n'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7)", 'ErrorSummaryDescription': "'instance-00000020' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000020.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:20.554 4076 3051054571192 MainThread ERROR nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Failed to change vm state of instance-00000020 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. 2021-09-10 16:54:20.554 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Got request to destroy instance 2021-09-10 16:54:20.554 4076 3051092584760 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:20.554 4076 3051092584760 MainThread INFO nova.compute.manager [-] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] VM Stopped (Lifecycle Event) 2021-09-10 16:54:20.569 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:20.576 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:20.576 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-f42a8587-25ab-4388-9c72-fe17cd07fdcc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:20.576 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.vmops [req-f42a8587-25ab-4388-9c72-fe17cd07fdcc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:20.600 4076 3051092582472 MainThread DEBUG nova.compute.manager [req-9ae33df7-1e5b-4c63-ad6c-bd83a83738f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Received event network-vif-plugged-e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:20.600 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [req-9ae33df7-1e5b-4c63-ad6c-bd83a83738f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:20.608 4076 3051092582472 MainThread DEBUG oslo_concurrency.lockutils [req-9ae33df7-1e5b-4c63-ad6c-bd83a83738f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:20.608 4076 3051092582472 MainThread DEBUG nova.compute.manager [req-9ae33df7-1e5b-4c63-ad6c-bd83a83738f0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Processing event network-vif-plugged-e0a6eaf9-527b-44d8-8e4e-5beaa55bf373 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:54:20.652 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Using config drive for instance 2021-09-10 16:54:20.652 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-f42a8587-25ab-4388-9c72-fe17cd07fdcc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:20.710 4076 3051092584760 MainThread INFO nova.compute.manager [req-f42a8587-25ab-4388-9c72-fe17cd07fdcc - - - - -] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:21.126 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '14BE202A-F2A7-4FAF-9912-F722563218E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22263), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:21.310 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:21.310 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:21.310 4076 3051092584760 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:21.310 4076 3051092584760 MainThread INFO nova.compute.manager [-] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] VM Stopped (Lifecycle Event) 2021-09-10 16:54:21.340 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-742be031-8ebb-435c-b9fa-303014b88718 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:21.340 4076 3051092584760 MainThread DEBUG nova.virt.hyperv.vmops [req-742be031-8ebb-435c-b9fa-303014b88718 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:21.380 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '14BE202A-F2A7-4FAF-9912-F722563218E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=159430), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:21.380 4076 3051054571368 MainThread DEBUG os_win._utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.061818361282349 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:21.404 4076 3051092584760 MainThread DEBUG nova.compute.manager [req-742be031-8ebb-435c-b9fa-303014b88718 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:21.420 4076 3051092583000 MainThread DEBUG nova.objects.instance [-] Lazy-loading 'migration_context' on Instance uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:54:21.442 4076 3051092584760 MainThread INFO nova.compute.manager [req-742be031-8ebb-435c-b9fa-303014b88718 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:21.751 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Creating config drive at C:\openstack\instances\instance-00000024\configdrive.iso 2021-09-10 16:54:21.812 4076 3051054569256 MainThread DEBUG oslo_concurrency.processutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000024\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkybqmxy3 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:54:22.131 4076 3051054569256 MainThread DEBUG oslo_concurrency.processutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000024\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkybqmxy3" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:54:22.398 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '2111FEA2-D794-4276-92F9-0EBD55C26A2A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15846), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:22.547 4076 3051054571368 MainThread ERROR os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000023\' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000023\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)instance-000000239952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2111FEA2-D794-4276-92F9-0EBD55C26A2A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112622), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\nNot enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)\n\n'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC)", 'ErrorSummaryDescription': "'instance-00000023' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000023.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:22.547 4076 3051054571368 MainThread ERROR nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Failed to change vm state of instance-00000023 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. 2021-09-10 16:54:22.547 4076 3051054571368 MainThread INFO nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Got request to destroy instance 2021-09-10 16:54:22.562 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Setting boot order for instance: instance-00000024: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:54:22.562 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:22.562 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:22.562 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:22.562 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] VM Stopped (Lifecycle Event) 2021-09-10 16:54:22.591 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-edc7120a-c076-491b-b1d9-93fc6a7590a3 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:22.591 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-edc7120a-c076-491b-b1d9-93fc6a7590a3 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:22.607 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:54:22.639 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-edc7120a-c076-491b-b1d9-93fc6a7590a3 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:22.639 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'F4B2736F-CD61-4BFB-9F4B-25DBA364A11F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16398), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:22.672 4076 3051054567848 MainThread INFO nova.compute.manager [req-edc7120a-c076-491b-b1d9-93fc6a7590a3 - - - - -] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:22.784 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F4B2736F-CD61-4BFB-9F4B-25DBA364A11F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116332), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:22.784 4076 3051054569256 MainThread DEBUG os_win._utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17679166793823242 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:22.784 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:22.784 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:22.784 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:22.784 4076 3051092584760 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:22.828 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-f799dccf-8026-4b4e-ac4c-757cbfa152b5 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:22.828 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-f799dccf-8026-4b4e-ac4c-757cbfa152b5 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:22.828 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:22.828 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:22.828 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:22.852 4076 3051054571192 MainThread DEBUG os_win.utils.compute.vmutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Successfully changed vm state of instance-00000020 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:22.852 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Successfully changed state of VM instance-00000020 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:22.859 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-f799dccf-8026-4b4e-ac4c-757cbfa152b5 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:22.893 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Waiting for WMI job: {'InstanceID': '210152E6-B871-47AD-93DD-C2B2F9523EB8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15798), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:22.908 4076 3051054567848 MainThread INFO nova.compute.manager [req-f799dccf-8026-4b4e-ac4c-757cbfa152b5 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:23.000 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] WMI job succeeded: {'InstanceID': '210152E6-B871-47AD-93DD-C2B2F9523EB8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30562), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:23.000 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:23.000 4076 3051054571192 MainThread DEBUG os_win.utils.pathutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Removing directory: C:\openstack\instances\instance-00000020 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:23.000 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:23.014 4076 3051054571192 MainThread ERROR nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Traceback (most recent call last): 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(*args, **kwargs) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] admin_password, network_info, block_device_info) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(self, *args, **kwds) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] should_plug_vifs=False) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'req_state': req_state}) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] time.sleep(sleep_time) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return f(*args, **kwargs) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return self._wait_for_job(job_path) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] error_desc=err_desc) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During handling of the above exception, another exception occurred: 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Traceback (most recent call last): 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] yield resources 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] accel_info=accel_info) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(*args, **kwargs) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] admin_password, network_info, block_device_info) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(self, *args, **kwds) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] should_plug_vifs=False) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'req_state': req_state}) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] time.sleep(sleep_time) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return f(*args, **kwargs) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return self._wait_for_job(job_path) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] error_desc=err_desc) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:23.014 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:23.014 4076 3051054571192 MainThread INFO nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Terminating instance 2021-09-10 16:54:23.014 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:23.014 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Got request to destroy instance 2021-09-10 16:54:23.033 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:23.033 4076 3051054571192 MainThread DEBUG os_brick.initiator.windows.iscsi [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790', 'target_lun': 0, 'volume_id': 'f5252e88-87bd-471e-b572-a8d70f3c3790', 'auth_method': 'CHAP', 'auth_username': 'nVSjUhEE86QfhZaDKiyx', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:54:23.033 4076 3051054571192 MainThread DEBUG os_win.utils.storage.diskutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:54:23.086 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager.detach_volume..do_detach_volume" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:23.102 4076 3051054567848 MainThread INFO nova.compute.manager [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Detaching volume 6fe38b5b-f580-4b49-8822-02f1519e6936 2021-09-10 16:54:23.156 4076 3051054571192 MainThread DEBUG os_win.utils.storage.diskutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:54:23.156 4076 3051054571192 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 16:54:23.185 4076 3051054567848 MainThread INFO nova.virt.block_device [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Attempting to driver detach volume 6fe38b5b-f580-4b49-8822-02f1519e6936 from mountpoint /dev/sdb 2021-09-10 16:54:23.220 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.volumeops [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Detaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936', 'target_lun': 0, 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'auth_method': 'CHAP', 'auth_username': 'cLGGMECT57NdrLs34ud4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '9107836a-481d-4cb5-9c7d-6432f4cd5309', 'attached_at': '', 'detached_at': '', 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'serial': '6fe38b5b-f580-4b49-8822-02f1519e6936'} from instance-0000001f detach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:128 2021-09-10 16:54:23.220 4076 3051054571192 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-f5252e88-87bd-471e-b572-a8d70f3c3790 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 16:54:23.220 4076 3051054571192 MainThread DEBUG os_brick.initiator.windows.iscsi [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] <== disconnect_volume: return (187ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:54:23.220 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:23.220 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "instance-00000020" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:23.220 4076 3051054571192 MainThread INFO nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Took 0.20 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:23.220 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> get_volume_paths: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936', 'target_lun': 0, 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'auth_method': 'CHAP', 'auth_username': 'cLGGMECT57NdrLs34ud4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:54:23.249 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== get_volume_paths: return (29ms) ['\\\\.\\PhysicalDrive1'] trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:54:23.281 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.volumeops [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Detaching disk \\N-H1-808474-1\root\virtualization\v2:Msvm_DiskDrive.CreationClassName="Msvm_DiskDrive",DeviceID="Microsoft:353B3BE8-310C-4cf4-839E-4E1B14616136\\1",SystemCreationClassName="Msvm_ComputerSystem",SystemName="N-H1-808474-1" from instance: instance-0000001f detach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:292 2021-09-10 16:54:23.504 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936', 'target_lun': 0, 'volume_id': '6fe38b5b-f580-4b49-8822-02f1519e6936', 'auth_method': 'CHAP', 'auth_username': 'cLGGMECT57NdrLs34ud4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:54:23.504 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:54:23.593 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:54:23.622 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 16:54:23.671 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-6fe38b5b-f580-4b49-8822-02f1519e6936 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 16:54:23.685 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== disconnect_volume: return (180ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:54:23.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:23.797 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'E6F4D0F7-5F9E-414B-8F6B-8A64DAC0C0C5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15007), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:23.937 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E6F4D0F7-5F9E-414B-8F6B-8A64DAC0C0C5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115846), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:23.937 4076 3051054569256 MainThread DEBUG os_win._utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.329690933227539 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:23.937 4076 3051092582648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:23.937 4076 3051092582648 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:23.937 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:23.937 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:23.975 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-f37f9391-c2bf-40dc-906a-028900cf54db - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:23.975 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-f37f9391-c2bf-40dc-906a-028900cf54db - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:24.031 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-f37f9391-c2bf-40dc-906a-028900cf54db - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:24.050 4076 3051092582648 MainThread INFO nova.compute.manager [req-f37f9391-c2bf-40dc-906a-028900cf54db - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:24.544 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:24.562 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:24.562 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:24.562 4076 3051054571368 MainThread DEBUG os_win.utils.compute.vmutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Successfully changed vm state of instance-00000023 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:24.562 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Successfully changed state of VM instance-00000023 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:24.625 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '0957258B-86DB-4CC3-881E-3EED08B7072E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14570), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:24.746 4076 3051054571368 MainThread DEBUG os_win.utils.jobutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job succeeded: {'InstanceID': '0957258B-86DB-4CC3-881E-3EED08B7072E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28787), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:24.746 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:24.746 4076 3051054571368 MainThread DEBUG os_win.utils.pathutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Removing directory: C:\openstack\instances\instance-00000023 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:24.746 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:24.746 4076 3051054571368 MainThread ERROR nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Traceback (most recent call last): 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(*args, **kwargs) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] admin_password, network_info, block_device_info) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(self, *args, **kwds) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] should_plug_vifs=False) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'req_state': req_state}) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] time.sleep(sleep_time) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return f(*args, **kwargs) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return self._wait_for_job(job_path) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] error_desc=err_desc) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During handling of the above exception, another exception occurred: 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Traceback (most recent call last): 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] yield resources 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] accel_info=accel_info) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(*args, **kwargs) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] admin_password, network_info, block_device_info) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(self, *args, **kwds) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] should_plug_vifs=False) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'req_state': req_state}) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] time.sleep(sleep_time) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return f(*args, **kwargs) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return self._wait_for_job(job_path) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] error_desc=err_desc) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:24.746 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:24.746 4076 3051054571368 MainThread INFO nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Terminating instance 2021-09-10 16:54:24.746 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:24.746 4076 3051054571368 MainThread INFO nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Got request to destroy instance 2021-09-10 16:54:24.763 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.vmops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:24.763 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:24.763 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-00000023" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:24.763 4076 3051054571368 MainThread INFO nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:24.763 4076 3051054571368 MainThread DEBUG nova.compute.claims [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:24.763 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:24.950 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.hostops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:24.950 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '032D36AD-7953-491B-A964-D6ADB3A380D7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15690), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:24.967 4076 3051054571368 MainThread DEBUG nova.virt.hyperv.hostops [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:25.094 4076 3051092582648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:25.094 4076 3051092582648 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:25.094 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:25.094 4076 3051092585816 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:25.094 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '032D36AD-7953-491B-A964-D6ADB3A380D7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111611), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:25.094 4076 3051054569256 MainThread DEBUG os_win._utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.486924886703491 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:25.122 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-8c962fd2-ed9e-407e-84e7-f1c369cc26ec - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:25.122 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-8c962fd2-ed9e-407e-84e7-f1c369cc26ec - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:25.187 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-8c962fd2-ed9e-407e-84e7-f1c369cc26ec - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:25.187 4076 3051054571368 MainThread DEBUG nova.compute.provider_tree [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:25.208 4076 3051054571368 MainThread DEBUG nova.scheduler.client.report [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:25.208 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:25.208 4076 3051054571368 MainThread ERROR nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Traceback (most recent call last): 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(*args, **kwargs) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] admin_password, network_info, block_device_info) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(self, *args, **kwds) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] should_plug_vifs=False) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'req_state': req_state}) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] time.sleep(sleep_time) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return f(*args, **kwargs) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return self._wait_for_job(job_path) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] error_desc=err_desc) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] During handling of the above exception, another exception occurred: 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Traceback (most recent call last): 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] accel_info=accel_info) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(*args, **kwargs) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] admin_password, network_info, block_device_info) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return function(self, *args, **kwds) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] should_plug_vifs=False) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'req_state': req_state}) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] time.sleep(sleep_time) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self.force_reraise() 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] raise self.value 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return f(*args, **kwargs) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] return self._wait_for_job(job_path) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] error_desc=err_desc) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. 2021-09-10 16:54:25.208 4076 3051054571368 MainThread TRACE nova.compute.manager [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.compute.utils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Build of instance c466fddc-e13c-4586-a808-4e92dcac06cc was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000023' failed to start. Not enough memory in the system to start the virtual machine instance-00000023. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000023' failed to start. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Not enough memory in the system to start the virtual machine instance-00000023 with ram size 1024 megabytes. (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) 'instance-00000023' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9952E7FC-5CE2-4EBF-81B3-C23A02E9C3EC) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:25.217 4076 3051054571368 MainThread DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:54:25.217 4076 3051092582648 MainThread INFO nova.compute.manager [req-8c962fd2-ed9e-407e-84e7-f1c369cc26ec - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:25.429 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Received event network-changed-5beac292-790e-4830-8f59-e411516f08cc external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:25.429 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing instance network info cache due to event network-changed-5beac292-790e-4830-8f59-e411516f08cc. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:25.429 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:25.429 4076 3051092582648 MainThread DEBUG nova.network.neutron [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Refreshing network info cache for port 5beac292-790e-4830-8f59-e411516f08cc _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:26.126 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '5BD86E93-FFF4-4195-B3D7-D74D66B3BD9D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14782), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:26.270 4076 3051093913848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:26.270 4076 3051093913848 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:26.270 4076 3051093914200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:26.270 4076 3051093914200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:26.270 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5BD86E93-FFF4-4195-B3D7-D74D66B3BD9D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110812), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:26.279 4076 3051054569256 MainThread DEBUG os_win._utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6723921298980713 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:26.298 4076 3051093913848 MainThread DEBUG nova.compute.manager [req-50f38f0a-908d-4b8c-ba5e-29d9300e0ec6 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:26.298 4076 3051093913848 MainThread DEBUG nova.virt.hyperv.vmops [req-50f38f0a-908d-4b8c-ba5e-29d9300e0ec6 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:26.344 4076 3051093913848 MainThread DEBUG nova.compute.manager [req-50f38f0a-908d-4b8c-ba5e-29d9300e0ec6 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:26.373 4076 3051093913848 MainThread INFO nova.compute.manager [req-50f38f0a-908d-4b8c-ba5e-29d9300e0ec6 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:26.706 4076 3051054571368 MainThread DEBUG nova.network.neutron [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:26.725 4076 3051054571368 MainThread INFO nova.compute.manager [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: c466fddc-e13c-4586-a808-4e92dcac06cc] Took 1.52 seconds to deallocate network for instance. 2021-09-10 16:54:26.794 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:26.794 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:27.019 4076 3051054571368 MainThread INFO nova.scheduler.client.report [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Deleted allocations for instance c466fddc-e13c-4586-a808-4e92dcac06cc 2021-09-10 16:54:27.061 4076 3051054571368 MainThread DEBUG oslo_concurrency.lockutils [req-4b7967d0-32d0-47b8-8bc1-ef1f7575e815 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "c466fddc-e13c-4586-a808-4e92dcac06cc" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.750s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:27.297 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'A834DA53-4035-4E03-9F16-8D2EC304349B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14528), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:27.442 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A834DA53-4035-4E03-9F16-8D2EC304349B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110759), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:27.442 4076 3051054569256 MainThread DEBUG os_win._utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.834710597991943 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:27.451 4076 3051092586344 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:27.451 4076 3051092586344 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:27.451 4076 3051093913672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:27.451 4076 3051093913672 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:27.487 4076 3051092586344 MainThread DEBUG nova.compute.manager [req-9638cf78-106b-4295-8316-a65fc1df50d9 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:27.487 4076 3051092586344 MainThread DEBUG nova.virt.hyperv.vmops [req-9638cf78-106b-4295-8316-a65fc1df50d9 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:27.536 4076 3051092586344 MainThread DEBUG nova.compute.manager [req-9638cf78-106b-4295-8316-a65fc1df50d9 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:27.546 4076 3051092582648 MainThread DEBUG nova.network.neutron [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated VIF entry in instance network info cache for port 5beac292-790e-4830-8f59-e411516f08cc. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:27.546 4076 3051092582648 MainThread DEBUG nova.network.neutron [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:27.566 4076 3051092582648 MainThread DEBUG oslo_concurrency.lockutils [req-8fd6ef96-d99e-4e89-83ac-e4154fcb5ee5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:27.576 4076 3051092586344 MainThread INFO nova.compute.manager [req-9638cf78-106b-4295-8316-a65fc1df50d9 - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:27.793 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:27.913 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:27.913 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:54:28.170 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:28.170 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:54:28.458 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '0B8D0ECF-5554-41D4-A5AF-893FDB58275D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15601), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:28.576 4076 3051054569256 MainThread ERROR os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000024\' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000024\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)instance-000000249806B5F3-6C96-47C5-BEA2-B328B750536D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0B8D0ECF-5554-41D4-A5AF-893FDB58275D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116070), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\nNot enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)\n\n'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D)", 'ErrorSummaryDescription': "'instance-00000024' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000024.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:28.605 4076 3051054569256 MainThread ERROR nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Failed to change vm state of instance-00000024 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. 2021-09-10 16:54:28.605 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Got request to destroy instance 2021-09-10 16:54:28.605 4076 3051093917768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:28.605 4076 3051093917768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:28.605 4076 3051092582648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:28.605 4076 3051092582648 MainThread INFO nova.compute.manager [-] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] VM Stopped (Lifecycle Event) 2021-09-10 16:54:28.639 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-4812049c-b4a2-44f7-9108-cd4e56ee545d - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:28.639 4076 3051092582648 MainThread DEBUG nova.virt.hyperv.vmops [req-4812049c-b4a2-44f7-9108-cd4e56ee545d - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:28.684 4076 3051092582648 MainThread DEBUG nova.compute.manager [req-4812049c-b4a2-44f7-9108-cd4e56ee545d - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:28.715 4076 3051092582648 MainThread INFO nova.compute.manager [req-4812049c-b4a2-44f7-9108-cd4e56ee545d - - - - -] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:29.333 4076 3051054571192 MainThread INFO nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Took 6.11 seconds to detach 1 volumes for instance. 2021-09-10 16:54:29.333 4076 3051054571192 MainThread DEBUG nova.compute.claims [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:29.333 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:29.531 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:29.675 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:29.904 4076 3051054571192 MainThread DEBUG nova.compute.provider_tree [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:29.931 4076 3051054571192 MainThread DEBUG nova.scheduler.client.report [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:29.931 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:29.931 4076 3051054571192 MainThread ERROR nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Traceback (most recent call last): 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(*args, **kwargs) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] admin_password, network_info, block_device_info) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(self, *args, **kwds) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] should_plug_vifs=False) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'req_state': req_state}) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] time.sleep(sleep_time) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return f(*args, **kwargs) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return self._wait_for_job(job_path) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] error_desc=err_desc) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] During handling of the above exception, another exception occurred: 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Traceback (most recent call last): 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] accel_info=accel_info) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(*args, **kwargs) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] admin_password, network_info, block_device_info) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return function(self, *args, **kwds) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] should_plug_vifs=False) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'req_state': req_state}) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] time.sleep(sleep_time) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self.force_reraise() 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] raise self.value 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return f(*args, **kwargs) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] return self._wait_for_job(job_path) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] error_desc=err_desc) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. 2021-09-10 16:54:29.931 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.compute.utils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Build of instance f3fbc33e-18d4-47bf-b643-94d2edfcae5f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000020' failed to start. Not enough memory in the system to start the virtual machine instance-00000020. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000020' failed to start. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Not enough memory in the system to start the virtual machine instance-00000020 with ram size 1024 megabytes. (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) 'instance-00000020' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 30978065-CAA0-48E2-9DA4-C1446FB885A7) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:29.938 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:54:30.217 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:30.272 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:30.272 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:54:30.272 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:30.272 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:30.299 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:30.299 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:30.363 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:30.363 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:30.363 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:54:30.363 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:30.381 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:30.528 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:30.528 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:30.528 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:30.546 4076 3051054569256 MainThread DEBUG os_win.utils.compute.vmutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed vm state of instance-00000024 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:30.546 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed state of VM instance-00000024 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:30.576 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1489MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:54:30.576 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:30.612 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'D8F6C0AC-8861-4440-B47B-50AFDAC93417', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17374), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:30.733 4076 3051054569256 MainThread DEBUG os_win.utils.jobutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job succeeded: {'InstanceID': 'D8F6C0AC-8861-4440-B47B-50AFDAC93417', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30093), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:30.733 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:30.733 4076 3051054569256 MainThread DEBUG os_win.utils.pathutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Removing directory: C:\openstack\instances\instance-00000024 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:30.733 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:30.733 4076 3051054569256 MainThread ERROR nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Traceback (most recent call last): 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(*args, **kwargs) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] admin_password, network_info, block_device_info) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(self, *args, **kwds) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] should_plug_vifs=False) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'req_state': req_state}) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] time.sleep(sleep_time) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return f(*args, **kwargs) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return self._wait_for_job(job_path) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] error_desc=err_desc) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During handling of the above exception, another exception occurred: 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Traceback (most recent call last): 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] yield resources 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] accel_info=accel_info) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(*args, **kwargs) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] admin_password, network_info, block_device_info) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(self, *args, **kwds) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] should_plug_vifs=False) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'req_state': req_state}) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] time.sleep(sleep_time) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return f(*args, **kwargs) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return self._wait_for_job(job_path) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] error_desc=err_desc) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:30.733 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:30.748 4076 3051054569256 MainThread INFO nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Terminating instance 2021-09-10 16:54:30.748 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:30.748 4076 3051054569256 MainThread INFO nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Got request to destroy instance 2021-09-10 16:54:30.768 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:30.768 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:30.768 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000024" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:30.768 4076 3051054569256 MainThread INFO nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:30.768 4076 3051054569256 MainThread DEBUG nova.compute.claims [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:30.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f3fbc33e-18d4-47bf-b643-94d2edfcae5f has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6f3c13e4-15bf-4dd6-ad7e-7870384f5220 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:54:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:54:30.887 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:30.906 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:31.108 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:31.108 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:31.137 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:54:31.137 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.562s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:31.137 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:31.284 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:31.300 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.hostops [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:31.300 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:31.328 4076 3051054571192 MainThread INFO nova.compute.manager [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] [instance: f3fbc33e-18d4-47bf-b643-94d2edfcae5f] Took 1.39 seconds to deallocate network for instance. 2021-09-10 16:54:31.498 4076 3051054569256 MainThread DEBUG nova.compute.provider_tree [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.scheduler.client.report [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:31.569 4076 3051054569256 MainThread ERROR nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Traceback (most recent call last): 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(*args, **kwargs) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] admin_password, network_info, block_device_info) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(self, *args, **kwds) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] should_plug_vifs=False) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'req_state': req_state}) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] time.sleep(sleep_time) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return f(*args, **kwargs) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return self._wait_for_job(job_path) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] error_desc=err_desc) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] During handling of the above exception, another exception occurred: 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Traceback (most recent call last): 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] accel_info=accel_info) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(*args, **kwargs) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] admin_password, network_info, block_device_info) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return function(self, *args, **kwds) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] should_plug_vifs=False) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'req_state': req_state}) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] time.sleep(sleep_time) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self.force_reraise() 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] raise self.value 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return f(*args, **kwargs) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] return self._wait_for_job(job_path) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] error_desc=err_desc) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. 2021-09-10 16:54:31.569 4076 3051054569256 MainThread TRACE nova.compute.manager [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.compute.utils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Build of instance 6f3c13e4-15bf-4dd6-ad7e-7870384f5220 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000024' failed to start. Not enough memory in the system to start the virtual machine instance-00000024. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000024' failed to start. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Not enough memory in the system to start the virtual machine instance-00000024 with ram size 1024 megabytes. (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) 'instance-00000024' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9806B5F3-6C96-47C5-BEA2-B328B750536D) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:31.569 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:31.584 4076 3051054569256 MainThread DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:54:31.632 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:31.632 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:54:31.632 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:54:31.647 4076 3051054571192 MainThread INFO nova.scheduler.client.report [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Deleted allocations for instance f3fbc33e-18d4-47bf-b643-94d2edfcae5f 2021-09-10 16:54:31.673 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-7e06ecf1-e4b6-4913-b62a-aef6ef7ada8b 4826ecf435ad43b6984adee7a94b6ebc 7199a32f43ef47a1940566d570e22783 - default default] Lock "f3fbc33e-18d4-47bf-b643-94d2edfcae5f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 46.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:32.752 4076 3051054569256 MainThread DEBUG nova.network.neutron [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:32.762 4076 3051054569256 MainThread INFO nova.compute.manager [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 6f3c13e4-15bf-4dd6-ad7e-7870384f5220] Took 1.17 seconds to deallocate network for instance. 2021-09-10 16:54:33.085 4076 3051054569256 MainThread INFO nova.scheduler.client.report [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Deleted allocations for instance 6f3c13e4-15bf-4dd6-ad7e-7870384f5220 2021-09-10 16:54:33.091 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:33.121 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-4c12700e-58e1-44d6-83bf-81c24936952b d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "6f3c13e4-15bf-4dd6-ad7e-7870384f5220" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:33.184 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:54:33.385 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:33.385 4076 3051053528520 MainThread DEBUG nova.virt.hardware [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:54:33.385 4076 3051053528520 MainThread INFO nova.compute.claims [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Claim successful on node n-h1-808474-1 2021-09-10 16:54:33.583 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.hostops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:33.601 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.hostops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:33.787 4076 3051053528520 MainThread DEBUG nova.compute.provider_tree [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:33.811 4076 3051053528520 MainThread DEBUG nova.scheduler.client.report [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:33.811 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:33.811 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:54:33.903 4076 3051053528520 MainThread DEBUG nova.compute.utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:54:33.903 4076 3051054569256 GreenThread-24 DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:54:33.903 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:54:33.937 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:54:34.223 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:54:34.233 4076 3051053528520 MainThread INFO nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Spawning new instance 2021-09-10 16:54:34.233 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:34.233 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:34.256 4076 3051053528520 MainThread DEBUG nova.block_device [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:54:34.256 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:34.256 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:34.269 4076 3051053528520 MainThread DEBUG os_win.utils.pathutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Creating directory: C:\openstack\instances\instance-00000025 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:54:34.269 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000025\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:54:34.307 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Resizing VHD C:\openstack\instances\instance-00000025\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:54:34.624 4076 3051054569256 GreenThread-24 DEBUG nova.policy [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a884270e965c4547831a05eb6d766a49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8104d9820d2e4012a4147616bf84cac4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:54:35.085 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:35.118 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:54:35.309 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:35.309 4076 3051053528872 MainThread DEBUG nova.virt.hardware [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:54:35.309 4076 3051053528872 MainThread INFO nova.compute.claims [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Claim successful on node n-h1-808474-1 2021-09-10 16:54:35.477 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:35.477 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:35.715 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:35.738 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:35.738 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:35.738 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:54:35.843 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:54:35.843 4076 3051053528696 GreenThread-25 DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:54:35.843 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:54:35.875 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:54:36.055 4076 3051053528696 GreenThread-25 DEBUG nova.policy [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73ac9100b70e4fa1b6ad7751b223e5ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '778686f367c642f39349bb208103a3ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:54:36.109 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:54:36.109 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Spawning new instance 2021-09-10 16:54:36.109 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:36.109 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:36.109 4076 3051053528872 MainThread DEBUG nova.block_device [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:54:36.133 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:36.133 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:36.143 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Creating directory: C:\openstack\instances\instance-00000026 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:54:36.153 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000026\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:54:36.186 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Resizing VHD C:\openstack\instances\instance-00000026\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:54:36.466 4076 3051054567848 MainThread DEBUG nova.objects.instance [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lazy-loading 'flavor' on Instance uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:54:36.515 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-1e6afebd-9a4e-45b0-be48-ad98f74b4d26 beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager.detach_volume..do_detach_volume" :: held 13.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:37.267 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Successfully created port: 568ce78c-39c4-4d1f-9326-42dc1beb8796 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:39.871 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully created port: 299081df-ec8d-4d29-b4a3-234b773d5d47 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:40.003 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Successfully updated port: 568ce78c-39c4-4d1f-9326-42dc1beb8796 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:40.043 4076 3051054569256 GreenThread-24 DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Acquired lock "refresh_cache-c6fa36bc-df54-4f59-82b7-4db06ead4dd7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:40.043 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:54:40.144 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:40.828 4076 3051054569256 GreenThread-24 DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Updating instance_info_cache with network_info: [{"id": "568ce78c-39c4-4d1f-9326-42dc1beb8796", "address": "fa:16:3e:04:be:bb", "network": {"id": "6a1373b1-59c8-4186-817b-9e6d041358e3", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-613717313-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8104d9820d2e4012a4147616bf84cac4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap568ce78c-39", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:40.871 4076 3051054569256 GreenThread-24 DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Releasing lock "refresh_cache-c6fa36bc-df54-4f59-82b7-4db06ead4dd7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:40.871 4076 3051054569256 GreenThread-24 DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Instance network_info: |[{"id": "568ce78c-39c4-4d1f-9326-42dc1beb8796", "address": "fa:16:3e:04:be:bb", "network": {"id": "6a1373b1-59c8-4186-817b-9e6d041358e3", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-613717313-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8104d9820d2e4012a4147616bf84cac4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap568ce78c-39", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:54:40.871 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Preparing to wait for external event network-vif-plugged-568ce78c-39c4-4d1f-9326-42dc1beb8796 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:40.871 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:40.871 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:40.871 4076 3051053528520 MainThread DEBUG os_win.utils.compute.vmutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Creating VM instance-00000025 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:54:41.734 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:42.200 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Received event network-changed-568ce78c-39c4-4d1f-9326-42dc1beb8796 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:42.200 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Refreshing instance network info cache due to event network-changed-568ce78c-39c4-4d1f-9326-42dc1beb8796. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:42.200 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-c6fa36bc-df54-4f59-82b7-4db06ead4dd7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:42.200 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Refreshing network info cache for port 568ce78c-39c4-4d1f-9326-42dc1beb8796 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:42.547 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully created port: 77ee43b1-729c-457b-b4d4-7e8ec575f2a2 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:42.887 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Updated VIF entry in instance network info cache for port 568ce78c-39c4-4d1f-9326-42dc1beb8796. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:42.887 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Updating instance_info_cache with network_info: [{"id": "568ce78c-39c4-4d1f-9326-42dc1beb8796", "address": "fa:16:3e:04:be:bb", "network": {"id": "6a1373b1-59c8-4186-817b-9e6d041358e3", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-613717313-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8104d9820d2e4012a4147616bf84cac4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap568ce78c-39", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:42.918 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-3969046f-be49-4c04-aa86-bb4139f1e88e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-c6fa36bc-df54-4f59-82b7-4db06ead4dd7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:45.147 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully created port: 00c438ec-edfa-4ec8-90f4-cb57289b0684 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:54:47.402 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully updated port: 299081df-ec8d-4d29-b4a3-234b773d5d47 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:47.818 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-changed-299081df-ec8d-4d29-b4a3-234b773d5d47 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:47.818 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing instance network info cache due to event network-changed-299081df-ec8d-4d29-b4a3-234b773d5d47. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:47.818 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:47.818 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing network info cache for port 299081df-ec8d-4d29-b4a3-234b773d5d47 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:48.121 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:48.449 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-b811f9f5-3844-48c1-94ed-a79a9d4b3f0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Received event network-vif-plugged-568ce78c-39c4-4d1f-9326-42dc1beb8796 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:48.449 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-b811f9f5-3844-48c1-94ed-a79a9d4b3f0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:48.449 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-b811f9f5-3844-48c1-94ed-a79a9d4b3f0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:48.449 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-b811f9f5-3844-48c1-94ed-a79a9d4b3f0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Processing event network-vif-plugged-568ce78c-39c4-4d1f-9326-42dc1beb8796 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:54:48.493 4076 3051053528520 MainThread INFO nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Using config drive for instance 2021-09-10 16:54:48.844 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:48.863 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-d42a936d-375f-44e5-ad45-3bb9375759b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:48.921 4076 3051053528520 MainThread INFO nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Creating config drive at C:\openstack\instances\instance-00000025\configdrive.iso 2021-09-10 16:54:48.959 4076 3051053528520 MainThread DEBUG oslo_concurrency.processutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000025\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpclth0ish execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:54:49.271 4076 3051053528520 MainThread DEBUG oslo_concurrency.processutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000025\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpclth0ish" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:54:49.466 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully updated port: 77ee43b1-729c-457b-b4d4-7e8ec575f2a2 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:49.669 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Setting boot order for instance: instance-00000025: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:54:49.716 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:54:49.730 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': '285D88AD-EE82-40A8-9C7F-86CF9E633237', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15205), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:49.874 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:49.874 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:49.874 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:49.874 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:49.874 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '285D88AD-EE82-40A8-9C7F-86CF9E633237', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112399), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:49.893 4076 3051053528520 MainThread DEBUG os_win._utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17716121673583984 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:49.908 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-e1a46c19-b40e-482f-8e79-d4feb8de2a03 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:49.908 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-e1a46c19-b40e-482f-8e79-d4feb8de2a03 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:49.952 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-e1a46c19-b40e-482f-8e79-d4feb8de2a03 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:49.982 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-changed-77ee43b1-729c-457b-b4d4-7e8ec575f2a2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:49.982 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing instance network info cache due to event network-changed-77ee43b1-729c-457b-b4d4-7e8ec575f2a2. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:49.982 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:49.982 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing network info cache for port 77ee43b1-729c-457b-b4d4-7e8ec575f2a2 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:49.982 4076 3051054567848 MainThread INFO nova.compute.manager [req-e1a46c19-b40e-482f-8e79-d4feb8de2a03 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:50.137 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:50.921 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': '92E241A3-5E60-44C3-BF73-212A4F2B99D8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14814), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:51.062 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:51.062 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:51.062 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:51.062 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:51.062 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '92E241A3-5E60-44C3-BF73-212A4F2B99D8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110929), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:51.062 4076 3051053528520 MainThread DEBUG os_win._utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3457982540130615 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:51.092 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cbed29d0-5e84-4d4e-8aaf-8b723856da82 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:51.092 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-cbed29d0-5e84-4d4e-8aaf-8b723856da82 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:51.107 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:51.125 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-a42148c4-22aa-47d9-ae9c-519c0c4f6cd7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:51.141 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-cbed29d0-5e84-4d4e-8aaf-8b723856da82 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:51.199 4076 3051054569256 MainThread INFO nova.compute.manager [req-cbed29d0-5e84-4d4e-8aaf-8b723856da82 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:51.367 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Successfully updated port: 00c438ec-edfa-4ec8-90f4-cb57289b0684 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:54:51.391 4076 3051053528696 GreenThread-25 DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Acquired lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:51.391 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:54:51.615 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:54:52.075 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': 'A9779F70-FB59-4519-9F4E-252676897131', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19309), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:52.137 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-changed-00c438ec-edfa-4ec8-90f4-cb57289b0684 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:54:52.137 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing instance network info cache due to event network-changed-00c438ec-edfa-4ec8-90f4-cb57289b0684. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:54:52.218 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:52.218 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:52.218 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:52.218 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:52.218 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A9779F70-FB59-4519-9F4E-252676897131', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112377), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:52.218 4076 3051053528520 MainThread DEBUG os_win._utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.502505302429199 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:52.246 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4d57f7b9-0ff7-4d8e-b264-57300985bb71 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:52.246 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4d57f7b9-0ff7-4d8e-b264-57300985bb71 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:52.291 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4d57f7b9-0ff7-4d8e-b264-57300985bb71 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:52.311 4076 3051054569256 MainThread INFO nova.compute.manager [req-4d57f7b9-0ff7-4d8e-b264-57300985bb71 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:53.263 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': 'C445A8B5-282D-4342-9F88-BDEA2A77D492', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16213), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:53.375 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:53.395 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:53.395 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:53.395 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:53.395 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C445A8B5-282D-4342-9F88-BDEA2A77D492', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111729), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:53.395 4076 3051053528520 MainThread DEBUG os_win._utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6793177127838135 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:53.427 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-024f575f-ed36-4eb2-a406-bf5e1894fd5f - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:53.427 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-024f575f-ed36-4eb2-a406-bf5e1894fd5f - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:53.451 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-024f575f-ed36-4eb2-a406-bf5e1894fd5f - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:53.521 4076 3051054569256 MainThread INFO nova.compute.manager [req-024f575f-ed36-4eb2-a406-bf5e1894fd5f - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:54.436 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': '7FB87873-425F-4E48-8388-792E8DFEE12C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18647), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:54.575 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:54.575 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:54.575 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:54.575 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:54.575 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7FB87873-425F-4E48-8388-792E8DFEE12C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=121909), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:54.575 4076 3051053528520 MainThread DEBUG os_win._utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.859031677246094 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:54:54.615 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-e3facf8e-7f4b-4739-b1c5-47f956f3769d - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:54.615 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-e3facf8e-7f4b-4739-b1c5-47f956f3769d - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:54.653 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-e3facf8e-7f4b-4739-b1c5-47f956f3769d - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:54.714 4076 3051054569256 MainThread INFO nova.compute.manager [req-e3facf8e-7f4b-4739-b1c5-47f956f3769d - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:54.834 4076 3051053528696 GreenThread-25 DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updating instance_info_cache with network_info: [{"id": "299081df-ec8d-4d29-b4a3-234b773d5d47", "address": "fa:16:3e:c3:7e:26", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.86", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap299081df-ec", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "77ee43b1-729c-457b-b4d4-7e8ec575f2a2", "address": "fa:16:3e:81:ef:16", "network": {"id": "2902a8f7-f5a1-41cb-848f-1eec8ed03eb3", "bridge": null, "label": "tempest-ServersTestMultiNic-1455612463", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap77ee43b1-72", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "00c438ec-edfa-4ec8-90f4-cb57289b0684", "address": "fa:16:3e:8c:bb:2c", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.240", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap00c438ec-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:54.871 4076 3051053528696 GreenThread-25 DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Releasing lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:54.871 4076 3051053528696 GreenThread-25 DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance network_info: |[{"id": "299081df-ec8d-4d29-b4a3-234b773d5d47", "address": "fa:16:3e:c3:7e:26", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.86", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap299081df-ec", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "77ee43b1-729c-457b-b4d4-7e8ec575f2a2", "address": "fa:16:3e:81:ef:16", "network": {"id": "2902a8f7-f5a1-41cb-848f-1eec8ed03eb3", "bridge": null, "label": "tempest-ServersTestMultiNic-1455612463", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap77ee43b1-72", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "00c438ec-edfa-4ec8-90f4-cb57289b0684", "address": "fa:16:3e:8c:bb:2c", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.240", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap00c438ec-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:54:54.871 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:54:54.871 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Refreshing network info cache for port 00c438ec-edfa-4ec8-90f4-cb57289b0684 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Preparing to wait for external event network-vif-plugged-299081df-ec8d-4d29-b4a3-234b773d5d47 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Preparing to wait for external event network-vif-plugged-77ee43b1-729c-457b-b4d4-7e8ec575f2a2 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Preparing to wait for external event network-vif-plugged-00c438ec-edfa-4ec8-90f4-cb57289b0684 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:54.871 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Creating VM instance-00000026 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:54:55.608 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': '61136ACB-F3B4-4BD0-BE85-B78322035546', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15261), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:55.750 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:54:55.750 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] VM Stopped (Lifecycle Event) 2021-09-10 16:54:55.750 4076 3051053528696 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:55.750 4076 3051053528696 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:55.750 4076 3051053528520 MainThread ERROR os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000025\' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A2010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000025\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)instance-00000025C75BBD54-C542-4C14-8073-CABAF9086A20%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '61136ACB-F3B4-4BD0-BE85-B78322035546', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111158), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 54, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 54, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\nNot enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)\n\n'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20)", 'ErrorSummaryDescription': "'instance-00000025' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000025.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:54:55.750 4076 3051053528520 MainThread ERROR nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Failed to change vm state of instance-00000025 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. 2021-09-10 16:54:55.750 4076 3051053528520 MainThread INFO nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Got request to destroy instance 2021-09-10 16:54:55.771 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-f724ea4f-07d8-4061-82f9-5c2bab0209b2 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:54:55.781 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-f724ea4f-07d8-4061-82f9-5c2bab0209b2 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:54:55.794 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:55.811 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-f724ea4f-07d8-4061-82f9-5c2bab0209b2 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:54:55.862 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:55.862 4076 3051054569256 MainThread INFO nova.compute.manager [req-f724ea4f-07d8-4061-82f9-5c2bab0209b2 - - - - -] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:54:55.933 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:54:56.083 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updated VIF entry in instance network info cache for port 00c438ec-edfa-4ec8-90f4-cb57289b0684. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:54:56.083 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updating instance_info_cache with network_info: [{"id": "299081df-ec8d-4d29-b4a3-234b773d5d47", "address": "fa:16:3e:c3:7e:26", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.86", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap299081df-ec", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "77ee43b1-729c-457b-b4d4-7e8ec575f2a2", "address": "fa:16:3e:81:ef:16", "network": {"id": "2902a8f7-f5a1-41cb-848f-1eec8ed03eb3", "bridge": null, "label": "tempest-ServersTestMultiNic-1455612463", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap77ee43b1-72", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "00c438ec-edfa-4ec8-90f4-cb57289b0684", "address": "fa:16:3e:8c:bb:2c", "network": {"id": "22036b3c-62ab-4fc4-9c35-6bc38af85735", "bridge": null, "label": "tempest-ServersTestMultiNic-1292035885", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.240", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap00c438ec-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:54:56.106 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-d9c22e8f-7c76-496b-b0f5-d626d2390994 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-60d9f1cb-95b4-472e-999a-d08935125ced" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:54:57.598 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:54:57.598 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:57.598 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:57.610 4076 3051053528520 MainThread DEBUG os_win.utils.compute.vmutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Successfully changed vm state of instance-00000025 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:54:57.610 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Successfully changed state of VM instance-00000025 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:54:57.650 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Waiting for WMI job: {'InstanceID': '8125FE12-BC90-4422-B13E-7EABEE557AD8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14654), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:54:57.772 4076 3051053528520 MainThread DEBUG os_win.utils.jobutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] WMI job succeeded: {'InstanceID': '8125FE12-BC90-4422-B13E-7EABEE557AD8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28159), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:54:57.772 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:57.772 4076 3051053528520 MainThread DEBUG os_win.utils.pathutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Removing directory: C:\openstack\instances\instance-00000025 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:54:57.772 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:57.772 4076 3051053528520 MainThread ERROR nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Traceback (most recent call last): 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(*args, **kwargs) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] admin_password, network_info, block_device_info) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(self, *args, **kwds) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] should_plug_vifs=False) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'req_state': req_state}) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] time.sleep(sleep_time) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return f(*args, **kwargs) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return self._wait_for_job(job_path) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] error_desc=err_desc) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During handling of the above exception, another exception occurred: 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Traceback (most recent call last): 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] yield resources 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] accel_info=accel_info) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(*args, **kwargs) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] admin_password, network_info, block_device_info) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(self, *args, **kwds) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] should_plug_vifs=False) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'req_state': req_state}) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] time.sleep(sleep_time) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return f(*args, **kwargs) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return self._wait_for_job(job_path) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] error_desc=err_desc) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:57.772 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:57.786 4076 3051053528520 MainThread INFO nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Terminating instance 2021-09-10 16:54:57.786 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:54:57.786 4076 3051053528520 MainThread INFO nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Got request to destroy instance 2021-09-10 16:54:57.803 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:54:57.803 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:57.803 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "instance-00000025" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:57.803 4076 3051053528520 MainThread INFO nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:54:57.803 4076 3051053528520 MainThread DEBUG nova.compute.claims [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:54:57.803 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:54:57.986 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.hostops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:54:58.022 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.hostops [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:54:58.215 4076 3051053528520 MainThread DEBUG nova.compute.provider_tree [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.scheduler.client.report [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:54:58.246 4076 3051053528520 MainThread ERROR nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Traceback (most recent call last): 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(*args, **kwargs) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] admin_password, network_info, block_device_info) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(self, *args, **kwds) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] should_plug_vifs=False) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'req_state': req_state}) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] time.sleep(sleep_time) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return f(*args, **kwargs) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return self._wait_for_job(job_path) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] error_desc=err_desc) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] During handling of the above exception, another exception occurred: 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Traceback (most recent call last): 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] accel_info=accel_info) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(*args, **kwargs) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] admin_password, network_info, block_device_info) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return function(self, *args, **kwds) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] should_plug_vifs=False) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'req_state': req_state}) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] time.sleep(sleep_time) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self.force_reraise() 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] raise self.value 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return f(*args, **kwargs) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] return self._wait_for_job(job_path) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] error_desc=err_desc) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. 2021-09-10 16:54:58.246 4076 3051053528520 MainThread TRACE nova.compute.manager [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.compute.utils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Build of instance c6fa36bc-df54-4f59-82b7-4db06ead4dd7 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000025' failed to start. Not enough memory in the system to start the virtual machine instance-00000025. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000025' failed to start. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Not enough memory in the system to start the virtual machine instance-00000025 with ram size 1024 megabytes. (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) 'instance-00000025' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C75BBD54-C542-4C14-8073-CABAF9086A20) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:54:58.246 4076 3051053528520 MainThread DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:00.649 4076 3051053528520 MainThread DEBUG nova.network.neutron [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:00.702 4076 3051053528520 MainThread INFO nova.compute.manager [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] [instance: c6fa36bc-df54-4f59-82b7-4db06ead4dd7] Took 2.45 seconds to deallocate network for instance. 2021-09-10 16:55:01.042 4076 3051053528520 MainThread INFO nova.scheduler.client.report [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Deleted allocations for instance c6fa36bc-df54-4f59-82b7-4db06ead4dd7 2021-09-10 16:55:01.099 4076 3051053528520 MainThread DEBUG oslo_concurrency.lockutils [req-3f65a557-26d6-4962-aded-bf4250af178b a884270e965c4547831a05eb6d766a49 8104d9820d2e4012a4147616bf84cac4 - default default] Lock "c6fa36bc-df54-4f59-82b7-4db06ead4dd7" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 28.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:05.214 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-7b1e7994-2dc3-444c-bf18-fd4a2010d499 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-vif-plugged-299081df-ec8d-4d29-b4a3-234b773d5d47 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:05.214 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-7b1e7994-2dc3-444c-bf18-fd4a2010d499 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:05.214 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-7b1e7994-2dc3-444c-bf18-fd4a2010d499 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:05.214 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-7b1e7994-2dc3-444c-bf18-fd4a2010d499 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Processing event network-vif-plugged-299081df-ec8d-4d29-b4a3-234b773d5d47 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:05.987 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:05.998 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:06.207 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:06.230 4076 3051054567848 MainThread DEBUG nova.virt.hardware [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:06.230 4076 3051054567848 MainThread INFO nova.compute.claims [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Claim successful on node n-h1-808474-1 2021-09-10 16:55:06.407 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:06.437 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:06.644 4076 3051054567848 MainThread DEBUG nova.compute.provider_tree [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:06.658 4076 3051054567848 MainThread DEBUG nova.scheduler.client.report [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:06.658 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:06.658 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:06.815 4076 3051054567848 MainThread DEBUG nova.compute.utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:06.815 4076 3051092583000 GreenThread-26 DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:06.815 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:06.828 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:06.987 4076 3051054567848 MainThread INFO nova.virt.block_device [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Booting with volume 52401085-77e7-4280-b78c-bad720b115d8 at /dev/sda 2021-09-10 16:55:07.090 4076 3051054567848 MainThread DEBUG os_brick.utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:55:07.090 4076 3051054567848 MainThread DEBUG os_brick.utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:55:07.090 4076 3051054567848 MainThread DEBUG nova.virt.block_device [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Updating existing volume attachment record: ebfe0dab-b6c9-4ce3-a109-c0466d1b90aa _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:55:07.222 4076 3051092583000 GreenThread-26 DEBUG nova.policy [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'beffe9ba8529426fbb27e549254bd679', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91237ebda1b54d2dbc761e48fdd93516', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:09.465 4076 3051053528696 MainThread DEBUG nova.compute.manager [req-5f704775-8276-4586-a4a6-8473acd615d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-vif-plugged-00c438ec-edfa-4ec8-90f4-cb57289b0684 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:09.465 4076 3051053528696 MainThread DEBUG oslo_concurrency.lockutils [req-5f704775-8276-4586-a4a6-8473acd615d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:09.465 4076 3051053528696 MainThread DEBUG oslo_concurrency.lockutils [req-5f704775-8276-4586-a4a6-8473acd615d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:09.465 4076 3051053528696 MainThread DEBUG nova.compute.manager [req-5f704775-8276-4586-a4a6-8473acd615d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Processing event network-vif-plugged-00c438ec-edfa-4ec8-90f4-cb57289b0684 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:10.324 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Successfully created port: 1e4619a1-db47-424e-b019-df43c556540c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:12.772 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:12.772 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Spawning new instance 2021-09-10 16:55:12.787 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:12.787 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:12.787 4076 3051054567848 MainThread DEBUG nova.block_device [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:12.913 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Successfully updated port: 1e4619a1-db47-424e-b019-df43c556540c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:12.928 4076 3051092583000 GreenThread-26 DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Acquired lock "refresh_cache-5bd3ad21-3959-40c2-8698-b816b7176efc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:12.928 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:13.072 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:13.089 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:13.161 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:13.217 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:13.217 4076 3051053529224 MainThread DEBUG nova.virt.hardware [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:13.233 4076 3051053529224 MainThread INFO nova.compute.claims [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Claim successful on node n-h1-808474-1 2021-09-10 16:55:13.342 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.hostops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:13.375 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.hostops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:13.546 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-efdd5eed-f0e1-4814-bfce-42b052a0cd74 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Received event network-vif-plugged-77ee43b1-729c-457b-b4d4-7e8ec575f2a2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:13.546 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-efdd5eed-f0e1-4814-bfce-42b052a0cd74 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:13.569 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-efdd5eed-f0e1-4814-bfce-42b052a0cd74 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:13.569 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-efdd5eed-f0e1-4814-bfce-42b052a0cd74 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Processing event network-vif-plugged-77ee43b1-729c-457b-b4d4-7e8ec575f2a2 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:13.589 4076 3051053529224 MainThread DEBUG nova.compute.provider_tree [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:13.605 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Using config drive for instance 2021-09-10 16:55:13.647 4076 3051053529224 MainThread DEBUG nova.scheduler.client.report [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:13.647 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:13.647 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:13.746 4076 3051053529224 MainThread DEBUG nova.compute.utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:13.746 4076 3051053528696 GreenThread-27 DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:13.746 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:13.777 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:13.818 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Creating config drive at C:\openstack\instances\instance-00000026\configdrive.iso 2021-09-10 16:55:13.870 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000026\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplu9xcxhy execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:14.002 4076 3051092583000 GreenThread-26 DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Updating instance_info_cache with network_info: [{"id": "1e4619a1-db47-424e-b019-df43c556540c", "address": "fa:16:3e:14:ad:7c", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e4619a1-db", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:14.002 4076 3051053528696 GreenThread-27 DEBUG nova.policy [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '82eb2bcce085415e8b748990e15c58a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4ad3c7e6a94d4d6ca45d20d801674ca6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:14.027 4076 3051092583000 GreenThread-26 DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Releasing lock "refresh_cache-5bd3ad21-3959-40c2-8698-b816b7176efc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:14.027 4076 3051092583000 GreenThread-26 DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Instance network_info: |[{"id": "1e4619a1-db47-424e-b019-df43c556540c", "address": "fa:16:3e:14:ad:7c", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e4619a1-db", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:14.027 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Preparing to wait for external event network-vif-plugged-1e4619a1-db47-424e-b019-df43c556540c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:14.027 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.027 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:14.050 4076 3051054567848 MainThread DEBUG os_win.utils.compute.vmutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Creating VM instance-00000027 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:14.050 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Received event network-changed-1e4619a1-db47-424e-b019-df43c556540c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:14.050 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Refreshing instance network info cache due to event network-changed-1e4619a1-db47-424e-b019-df43c556540c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:14.064 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-5bd3ad21-3959-40c2-8698-b816b7176efc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:14.064 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Refreshing network info cache for port 1e4619a1-db47-424e-b019-df43c556540c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:14.139 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.167 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000026\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplu9xcxhy" returned: 0 in 0.296s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:14.183 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:14.199 4076 3051053529224 MainThread INFO nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Spawning new instance 2021-09-10 16:55:14.199 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:14.199 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG nova.block_device [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG os_win.utils.pathutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Creating directory: C:\openstack\instances\instance-00000028 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:55:14.215 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000028\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:55:14.255 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Resizing VHD C:\openstack\instances\instance-00000028\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:55:14.326 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.344 4076 3051053530104 MainThread DEBUG nova.virt.hardware [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:14.344 4076 3051053530104 MainThread INFO nova.compute.claims [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Claim successful on node n-h1-808474-1 2021-09-10 16:55:14.496 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:14.512 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:14.512 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.volumeops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8', 'target_lun': 0, 'volume_id': '52401085-77e7-4280-b78c-bad720b115d8', 'auth_method': 'CHAP', 'auth_username': 'Kn97TpVi2pyUkEmZbVJX', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '5bd3ad21-3959-40c2-8698-b816b7176efc', 'attached_at': '', 'detached_at': '', 'volume_id': '52401085-77e7-4280-b78c-bad720b115d8', 'serial': '52401085-77e7-4280-b78c-bad720b115d8'} to instance-00000027 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:55:14.512 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8', 'target_lun': 0, 'volume_id': '52401085-77e7-4280-b78c-bad720b115d8', 'auth_method': 'CHAP', 'auth_username': 'Kn97TpVi2pyUkEmZbVJX', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:55:14.512 4076 3051054567848 MainThread INFO os_brick.initiator.windows.iscsi [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:55:14.512 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:55:14.589 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Setting boot order for instance: instance-00000026: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:14.639 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:14.687 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '4957AD22-F67C-4B2A-82EA-8D3B5F0B84B5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22124), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:14.774 4076 3051053530104 MainThread DEBUG nova.compute.provider_tree [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:14.798 4076 3051053530104 MainThread DEBUG nova.scheduler.client.report [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:14.798 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:14.798 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:14.903 4076 3051053530104 MainThread DEBUG nova.compute.utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:14.903 4076 3051053529048 GreenThread-28 DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:14.903 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:14.918 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:14.918 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:14.918 4076 3051053529576 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:14.918 4076 3051053529576 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:14.939 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:14.959 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4957AD22-F67C-4B2A-82EA-8D3B5F0B84B5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=190238), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:14.959 4076 3051053528872 MainThread DEBUG os_win._utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.3200087547302246 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:14.974 4076 3051053529576 MainThread DEBUG nova.compute.manager [req-a960c7ea-72f9-4668-bed7-78fef989d381 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:14.974 4076 3051053529576 MainThread DEBUG nova.virt.hyperv.vmops [req-a960c7ea-72f9-4668-bed7-78fef989d381 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:15.030 4076 3051053529576 MainThread DEBUG nova.compute.manager [req-a960c7ea-72f9-4668-bed7-78fef989d381 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:15.072 4076 3051053529576 MainThread INFO nova.compute.manager [req-a960c7ea-72f9-4668-bed7-78fef989d381 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:15.177 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:15.177 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Spawning new instance 2021-09-10 16:55:15.198 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:15.198 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:15.198 4076 3051053530104 MainThread DEBUG nova.block_device [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:15.211 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:15.211 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:15.220 4076 3051053530104 MainThread DEBUG os_win.utils.pathutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating directory: C:\openstack\instances\instance-00000029 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:55:15.220 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000029\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:55:15.247 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Resizing VHD C:\openstack\instances\instance-00000029\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:55:15.363 4076 3051053529048 GreenThread-28 DEBUG nova.policy [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd33d11497a91483d93fe0ed8f5c26603', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'efc5deaa89294889821f303976e2d776', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:15.490 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Updated VIF entry in instance network info cache for port 1e4619a1-db47-424e-b019-df43c556540c. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:15.490 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Updating instance_info_cache with network_info: [{"id": "1e4619a1-db47-424e-b019-df43c556540c", "address": "fa:16:3e:14:ad:7c", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e4619a1-db", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:15.506 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-7433b6a1-f704-4494-ad6b-07063cea5f51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-5bd3ad21-3959-40c2-8698-b816b7176efc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:15.803 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Successfully created port: 7419ede1-6b4f-46c3-b4f6-5325702963d3 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:15.975 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '70C0CDF5-41FD-4559-9717-D7212447BFC1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15551), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:16.230 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:16.230 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:16.230 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:16.230 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:16.255 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-efe966ef-25b1-476a-9222-7a8b5537743b - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:16.255 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-efe966ef-25b1-476a-9222-7a8b5537743b - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:16.255 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '70C0CDF5-41FD-4559-9717-D7212447BFC1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=194333), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:16.255 4076 3051053528872 MainThread DEBUG os_win._utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.6163253784179688 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:16.294 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-efe966ef-25b1-476a-9222-7a8b5537743b - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:16.356 4076 3051054569256 MainThread INFO nova.compute.manager [req-efe966ef-25b1-476a-9222-7a8b5537743b - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:17.296 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '0A9074BB-412A-4070-A02B-CD990803381F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20434), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:17.448 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Successfully created port: cd6ab33b-3f48-4910-aac6-7d9f710bee54 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:17.503 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Successfully updated port: 7419ede1-6b4f-46c3-b4f6-5325702963d3 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:17.530 4076 3051053528696 GreenThread-27 DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Acquired lock "refresh_cache-714fea12-52ae-4277-8f3d-b0db2b2ee548" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:17.530 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:17.558 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:17.558 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:17.558 4076 3051053529576 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:17.558 4076 3051053529576 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:17.558 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0A9074BB-412A-4070-A02B-CD990803381F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=196045), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:17.558 4076 3051053528872 MainThread DEBUG os_win._utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.919348955154419 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:17.615 4076 3051053529576 MainThread DEBUG nova.compute.manager [req-1da852f3-0e9f-4577-ab0b-fcbeaaeb59aa - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:17.615 4076 3051053529576 MainThread DEBUG nova.virt.hyperv.vmops [req-1da852f3-0e9f-4577-ab0b-fcbeaaeb59aa - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:17.662 4076 3051053529576 MainThread DEBUG nova.compute.manager [req-1da852f3-0e9f-4577-ab0b-fcbeaaeb59aa - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:17.662 4076 3051053529400 MainThread DEBUG nova.compute.manager [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Received event network-changed-7419ede1-6b4f-46c3-b4f6-5325702963d3 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:17.662 4076 3051053529400 MainThread DEBUG nova.compute.manager [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Refreshing instance network info cache due to event network-changed-7419ede1-6b4f-46c3-b4f6-5325702963d3. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:17.683 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:17.683 4076 3051053529576 MainThread INFO nova.compute.manager [req-1da852f3-0e9f-4577-ab0b-fcbeaaeb59aa - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:18.558 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '1175E72F-68C8-4E6F-A33C-0E8C10DCA001', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15682), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:18.818 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:18.818 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:18.818 4076 3051053529576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:18.818 4076 3051053529576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:18.824 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-08cfb002-bc53-4045-b0c6-6c035c95ebd3 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:18.824 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-08cfb002-bc53-4045-b0c6-6c035c95ebd3 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:18.847 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1175E72F-68C8-4E6F-A33C-0E8C10DCA001', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=187195), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:18.847 4076 3051053528872 MainThread DEBUG os_win._utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.207916021347046 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:18.880 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-08cfb002-bc53-4045-b0c6-6c035c95ebd3 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:18.917 4076 3051053528696 GreenThread-27 DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Updating instance_info_cache with network_info: [{"id": "7419ede1-6b4f-46c3-b4f6-5325702963d3", "address": "fa:16:3e:b7:05:82", "network": {"id": "fd33a796-d83b-4f71-bce3-ee0ffa82326d", "bridge": null, "label": "tempest-ServersTestManualDisk-1887493174-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "4ad3c7e6a94d4d6ca45d20d801674ca6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7419ede1-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:18.917 4076 3051054571192 MainThread INFO nova.compute.manager [req-08cfb002-bc53-4045-b0c6-6c035c95ebd3 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:18.951 4076 3051053528696 GreenThread-27 DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Releasing lock "refresh_cache-714fea12-52ae-4277-8f3d-b0db2b2ee548" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:18.951 4076 3051053528696 GreenThread-27 DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Instance network_info: |[{"id": "7419ede1-6b4f-46c3-b4f6-5325702963d3", "address": "fa:16:3e:b7:05:82", "network": {"id": "fd33a796-d83b-4f71-bce3-ee0ffa82326d", "bridge": null, "label": "tempest-ServersTestManualDisk-1887493174-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "4ad3c7e6a94d4d6ca45d20d801674ca6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7419ede1-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:18.951 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-714fea12-52ae-4277-8f3d-b0db2b2ee548" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:18.951 4076 3051053529400 MainThread DEBUG nova.network.neutron [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Refreshing network info cache for port 7419ede1-6b4f-46c3-b4f6-5325702963d3 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:18.951 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Preparing to wait for external event network-vif-plugged-7419ede1-6b4f-46c3-b4f6-5325702963d3 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:18.951 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:18.951 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:18.951 4076 3051053529224 MainThread DEBUG os_win.utils.compute.vmutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Creating VM instance-00000028 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:19.458 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Successfully updated port: cd6ab33b-3f48-4910-aac6-7d9f710bee54 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:19.465 4076 3051053529048 GreenThread-28 DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Acquired lock "refresh_cache-f0c24368-9e06-4126-bbb0-d52db1f2141b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:19.465 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:19.521 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:55:19.640 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:19.657 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:55:19.722 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== connect_volume: return (5210ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '36001405E2A79E69D48A426B99722B553'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:55:19.856 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': 'F9FC1151-AB9E-4D6F-B229-C8F8EA569BAC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15107), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:19.856 4076 3051053529752 MainThread DEBUG nova.compute.manager [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Received event network-changed-cd6ab33b-3f48-4910-aac6-7d9f710bee54 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:19.856 4076 3051053529752 MainThread DEBUG nova.compute.manager [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Refreshing instance network info cache due to event network-changed-cd6ab33b-3f48-4910-aac6-7d9f710bee54. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:19.887 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:20.122 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9FC1151-AB9E-4D6F-B229-C8F8EA569BAC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=195141), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:20.122 4076 3051053528872 MainThread DEBUG os_win._utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.482748031616211 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:20.146 4076 3051053528520 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:20.146 4076 3051053528520 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:20.146 4076 3051053529928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:20.146 4076 3051053529928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:20.167 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-8ef77d0d-4474-46da-914c-e284859f93c6 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:20.185 4076 3051053528520 MainThread DEBUG nova.virt.hyperv.vmops [req-8ef77d0d-4474-46da-914c-e284859f93c6 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:20.202 4076 3051053529400 MainThread DEBUG nova.network.neutron [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Updated VIF entry in instance network info cache for port 7419ede1-6b4f-46c3-b4f6-5325702963d3. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:20.202 4076 3051053529400 MainThread DEBUG nova.network.neutron [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Updating instance_info_cache with network_info: [{"id": "7419ede1-6b4f-46c3-b4f6-5325702963d3", "address": "fa:16:3e:b7:05:82", "network": {"id": "fd33a796-d83b-4f71-bce3-ee0ffa82326d", "bridge": null, "label": "tempest-ServersTestManualDisk-1887493174-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "4ad3c7e6a94d4d6ca45d20d801674ca6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7419ede1-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:20.220 4076 3051053528520 MainThread DEBUG nova.compute.manager [req-8ef77d0d-4474-46da-914c-e284859f93c6 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:20.237 4076 3051053529400 MainThread DEBUG oslo_concurrency.lockutils [req-9244356a-0654-446d-a8a3-09741145c617 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-714fea12-52ae-4277-8f3d-b0db2b2ee548" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:20.237 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:20.261 4076 3051053528520 MainThread INFO nova.compute.manager [req-8ef77d0d-4474-46da-914c-e284859f93c6 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:20.437 4076 3051053529048 GreenThread-28 DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Updating instance_info_cache with network_info: [{"id": "cd6ab33b-3f48-4910-aac6-7d9f710bee54", "address": "fa:16:3e:9c:b1:e1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcd6ab33b-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:20.467 4076 3051053529048 GreenThread-28 DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Releasing lock "refresh_cache-f0c24368-9e06-4126-bbb0-d52db1f2141b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:20.467 4076 3051053529048 GreenThread-28 DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Instance network_info: |[{"id": "cd6ab33b-3f48-4910-aac6-7d9f710bee54", "address": "fa:16:3e:9c:b1:e1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcd6ab33b-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:20.467 4076 3051053529752 MainThread DEBUG oslo_concurrency.lockutils [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f0c24368-9e06-4126-bbb0-d52db1f2141b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:20.467 4076 3051053529752 MainThread DEBUG nova.network.neutron [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Refreshing network info cache for port cd6ab33b-3f48-4910-aac6-7d9f710bee54 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:20.467 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Preparing to wait for external event network-vif-plugged-cd6ab33b-3f48-4910-aac6-7d9f710bee54 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:20.467 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:20.467 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:20.467 4076 3051053530104 MainThread DEBUG os_win.utils.compute.vmutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating VM instance-00000029 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:21.143 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '2DFEAEE8-E5AC-4B83-9363-8C818B7B0E55', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14408), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:21.344 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:21.372 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:21.403 4076 3051053529576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:21.403 4076 3051053529576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:21.403 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:21.403 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] VM Stopped (Lifecycle Event) 2021-09-10 16:55:21.427 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-626bf3b4-a31a-40dc-b49b-85ca52ce6998 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:21.427 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-626bf3b4-a31a-40dc-b49b-85ca52ce6998 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:21.427 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:21.443 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-00000026\' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-99158541966310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000026\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)instance-0000002625F0BA1F-0E32-4715-B976-991585419663%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2DFEAEE8-E5AC-4B83-9363-8C818B7B0E55', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=195155), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\nNot enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)\n\n'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663)", 'ErrorSummaryDescription': "'instance-00000026' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000026.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:21.443 4076 3051053528872 MainThread ERROR nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Failed to change vm state of instance-00000026 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. 2021-09-10 16:55:21.443 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Got request to destroy instance 2021-09-10 16:55:21.469 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-626bf3b4-a31a-40dc-b49b-85ca52ce6998 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:21.493 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:21.496 4076 3051054571192 MainThread DEBUG nova.virt.hardware [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:21.496 4076 3051054571192 MainThread INFO nova.compute.claims [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Claim successful on node n-h1-808474-1 2021-09-10 16:55:21.523 4076 3051054569256 MainThread INFO nova.compute.manager [req-626bf3b4-a31a-40dc-b49b-85ca52ce6998 - - - - -] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:21.624 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:21.652 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:21.761 4076 3051053529752 MainThread DEBUG nova.network.neutron [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Updated VIF entry in instance network info cache for port cd6ab33b-3f48-4910-aac6-7d9f710bee54. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:21.761 4076 3051053529752 MainThread DEBUG nova.network.neutron [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Updating instance_info_cache with network_info: [{"id": "cd6ab33b-3f48-4910-aac6-7d9f710bee54", "address": "fa:16:3e:9c:b1:e1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcd6ab33b-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:21.777 4076 3051053529752 MainThread DEBUG oslo_concurrency.lockutils [req-14d5f4d7-6b92-432b-bc7d-3018c85cc55c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f0c24368-9e06-4126-bbb0-d52db1f2141b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:21.872 4076 3051054571192 MainThread DEBUG nova.compute.provider_tree [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:21.887 4076 3051054571192 MainThread DEBUG nova.scheduler.client.report [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:21.887 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:21.887 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:21.959 4076 3051054571192 MainThread DEBUG nova.compute.utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:21.959 4076 3051092583000 GreenThread-29 DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:21.959 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:21.980 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:22.164 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:22.170 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Spawning new instance 2021-09-10 16:55:22.170 4076 3051092583000 GreenThread-29 DEBUG nova.policy [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '14929f631b514b1ba7de48ea6f7875b6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f12d8e551e4d4d599a26e3249f7560f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:22.189 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:22.189 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:22.189 4076 3051054571192 MainThread DEBUG nova.block_device [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:22.189 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:22.189 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:22.199 4076 3051054571192 MainThread DEBUG os_win.utils.pathutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Creating directory: C:\openstack\instances\instance-0000002a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:55:22.199 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:55:22.230 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Resizing VHD C:\openstack\instances\instance-0000002a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:55:23.156 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Successfully created port: 747c7e5c-4098-4229-aaa0-b81088d50f7f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:23.618 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:55:23.618 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:23.618 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:23.636 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Successfully changed vm state of instance-00000026 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:55:23.636 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Successfully changed state of VM instance-00000026 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:55:23.677 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '8D49855F-7375-4A25-A314-85395BF0DB5B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14246), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:23.800 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job succeeded: {'InstanceID': '8D49855F-7375-4A25-A314-85395BF0DB5B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31426), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:55:23.800 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:23.800 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Removing directory: C:\openstack\instances\instance-00000026 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:55:23.800 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:23.800 4076 3051053528872 MainThread ERROR nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Traceback (most recent call last): 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(*args, **kwargs) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] admin_password, network_info, block_device_info) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(self, *args, **kwds) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] should_plug_vifs=False) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'req_state': req_state}) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] time.sleep(sleep_time) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return f(*args, **kwargs) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return self._wait_for_job(job_path) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] error_desc=err_desc) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During handling of the above exception, another exception occurred: 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Traceback (most recent call last): 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] yield resources 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] accel_info=accel_info) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(*args, **kwargs) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] admin_password, network_info, block_device_info) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(self, *args, **kwds) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] should_plug_vifs=False) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'req_state': req_state}) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] time.sleep(sleep_time) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return f(*args, **kwargs) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return self._wait_for_job(job_path) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] error_desc=err_desc) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:23.800 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:23.811 4076 3051053528872 MainThread INFO nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Terminating instance 2021-09-10 16:55:23.811 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:55:23.811 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Got request to destroy instance 2021-09-10 16:55:23.828 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:55:23.828 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:23.828 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-00000026" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:23.828 4076 3051053528872 MainThread INFO nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:55:23.828 4076 3051053528872 MainThread DEBUG nova.compute.claims [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:55:23.828 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:23.946 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Successfully updated port: 747c7e5c-4098-4229-aaa0-b81088d50f7f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:23.970 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:23.980 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:23.980 4076 3051092583000 GreenThread-29 DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Acquired lock "refresh_cache-6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:23.980 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:24.012 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Received event network-changed-747c7e5c-4098-4229-aaa0-b81088d50f7f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:24.012 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Refreshing instance network info cache due to event network-changed-747c7e5c-4098-4229-aaa0-b81088d50f7f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:24.061 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:24.199 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:24.215 4076 3051053528872 MainThread ERROR nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Traceback (most recent call last): 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(*args, **kwargs) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] admin_password, network_info, block_device_info) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(self, *args, **kwds) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] should_plug_vifs=False) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'req_state': req_state}) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] time.sleep(sleep_time) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return f(*args, **kwargs) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return self._wait_for_job(job_path) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] error_desc=err_desc) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] During handling of the above exception, another exception occurred: 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Traceback (most recent call last): 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] accel_info=accel_info) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(*args, **kwargs) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] admin_password, network_info, block_device_info) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return function(self, *args, **kwds) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] should_plug_vifs=False) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'req_state': req_state}) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] time.sleep(sleep_time) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self.force_reraise() 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] raise self.value 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return f(*args, **kwargs) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] return self._wait_for_job(job_path) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] error_desc=err_desc) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. 2021-09-10 16:55:24.215 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Build of instance 60d9f1cb-95b4-472e-999a-d08935125ced was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000026' failed to start. Not enough memory in the system to start the virtual machine instance-00000026. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000026' failed to start. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Not enough memory in the system to start the virtual machine instance-00000026 with ram size 1024 megabytes. (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) 'instance-00000026' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 25F0BA1F-0E32-4715-B976-991585419663) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:55:24.215 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:55:24.239 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:24.533 4076 3051092583000 GreenThread-29 DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Updating instance_info_cache with network_info: [{"id": "747c7e5c-4098-4229-aaa0-b81088d50f7f", "address": "fa:16:3e:df:36:ae", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap747c7e5c-40", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:24.550 4076 3051092583000 GreenThread-29 DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Releasing lock "refresh_cache-6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:24.550 4076 3051092583000 GreenThread-29 DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Instance network_info: |[{"id": "747c7e5c-4098-4229-aaa0-b81088d50f7f", "address": "fa:16:3e:df:36:ae", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap747c7e5c-40", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:24.550 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Preparing to wait for external event network-vif-plugged-747c7e5c-4098-4229-aaa0-b81088d50f7f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:24.550 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:24.550 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:24.560 4076 3051054571192 MainThread DEBUG os_win.utils.compute.vmutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Creating VM instance-0000002a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:24.560 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:24.560 4076 3051053529048 MainThread DEBUG nova.network.neutron [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Refreshing network info cache for port 747c7e5c-4098-4229-aaa0-b81088d50f7f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:25.026 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:25.517 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:25.553 4076 3051053529048 MainThread DEBUG nova.network.neutron [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Updated VIF entry in instance network info cache for port 747c7e5c-4098-4229-aaa0-b81088d50f7f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:25.553 4076 3051053529048 MainThread DEBUG nova.network.neutron [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Updating instance_info_cache with network_info: [{"id": "747c7e5c-4098-4229-aaa0-b81088d50f7f", "address": "fa:16:3e:df:36:ae", "network": {"id": "2b8d2ab0-2056-4c92-9f13-38b73897c01b", "bridge": null, "label": "tempest-SecurityGroupsTestJSON-625655316-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f12d8e551e4d4d599a26e3249f7560f1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap747c7e5c-40", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:25.573 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [req-5dd457f8-6d33-436d-a819-7b929163fb14 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:26.699 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-79fbc6e2-de30-4c02-a104-b515e5bfbd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Received event network-vif-plugged-1e4619a1-db47-424e-b019-df43c556540c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:26.699 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-79fbc6e2-de30-4c02-a104-b515e5bfbd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:26.699 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-79fbc6e2-de30-4c02-a104-b515e5bfbd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:26.699 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-79fbc6e2-de30-4c02-a104-b515e5bfbd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Processing event network-vif-plugged-1e4619a1-db47-424e-b019-df43c556540c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:26.761 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Using config drive for instance 2021-09-10 16:55:26.823 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:27.022 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Creating config drive at C:\openstack\instances\instance-00000027\configdrive.iso 2021-09-10 16:55:27.059 4076 3051054567848 MainThread DEBUG oslo_concurrency.processutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000027\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpb_30il4n execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:27.240 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:27.310 4076 3051053528872 MainThread INFO nova.compute.manager [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: 60d9f1cb-95b4-472e-999a-d08935125ced] Took 3.06 seconds to deallocate network for instance. 2021-09-10 16:55:27.381 4076 3051054567848 MainThread DEBUG oslo_concurrency.processutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000027\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpb_30il4n" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:27.533 4076 3051053528872 MainThread INFO nova.scheduler.client.report [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Deleted allocations for instance 60d9f1cb-95b4-472e-999a-d08935125ced 2021-09-10 16:55:27.562 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-c5d82b6b-4ae8-4bf3-a526-ddb7a75ca802 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "60d9f1cb-95b4-472e-999a-d08935125ced" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 52.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:27.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:27.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:55:27.829 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Setting boot order for instance: instance-00000027: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:27.871 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:27.871 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:55:27.871 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:27.926 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'B93DD77C-51DE-455F-AD12-E0C2FD91805D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14703), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:28.059 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:28.059 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:28.059 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:28.059 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:28.089 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-754507cc-ec3b-46ef-8c35-be547e3e56b6 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:28.089 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-754507cc-ec3b-46ef-8c35-be547e3e56b6 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:28.120 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B93DD77C-51DE-455F-AD12-E0C2FD91805D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115815), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:28.120 4076 3051054567848 MainThread DEBUG os_win._utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.23177838325500488 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:28.142 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-754507cc-ec3b-46ef-8c35-be547e3e56b6 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:28.186 4076 3051092583000 MainThread INFO nova.compute.manager [req-754507cc-ec3b-46ef-8c35-be547e3e56b6 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:28.936 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-c0978f9e-ae08-40bc-9f37-4d14ddffc168 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Received event network-vif-plugged-7419ede1-6b4f-46c3-b4f6-5325702963d3 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:28.936 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-c0978f9e-ae08-40bc-9f37-4d14ddffc168 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:28.936 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-c0978f9e-ae08-40bc-9f37-4d14ddffc168 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:28.936 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-c0978f9e-ae08-40bc-9f37-4d14ddffc168 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Processing event network-vif-plugged-7419ede1-6b4f-46c3-b4f6-5325702963d3 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:28.975 4076 3051053529224 MainThread INFO nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Using config drive for instance 2021-09-10 16:55:29.009 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:29.043 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:29.043 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:55:29.043 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:29.136 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'A8A5ECC1-9E69-4E15-9375-D7D12D852D60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15259), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:29.172 4076 3051053529224 MainThread INFO nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Creating config drive at C:\openstack\instances\instance-00000028\configdrive.iso 2021-09-10 16:55:29.222 4076 3051053529224 MainThread DEBUG oslo_concurrency.processutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000028\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpje90iomm execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:29.401 4076 3051053529048 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:29.401 4076 3051053529048 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:29.401 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:29.401 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:29.401 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A8A5ECC1-9E69-4E15-9375-D7D12D852D60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=136164), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:29.401 4076 3051054567848 MainThread DEBUG os_win._utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.5129637718200684 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:29.432 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-c829be44-a4ee-434c-b2b5-c5239c86d7a7 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:29.432 4076 3051053529048 MainThread DEBUG nova.virt.hyperv.vmops [req-c829be44-a4ee-434c-b2b5-c5239c86d7a7 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:29.466 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-c829be44-a4ee-434c-b2b5-c5239c86d7a7 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:29.519 4076 3051053529048 MainThread INFO nova.compute.manager [req-c829be44-a4ee-434c-b2b5-c5239c86d7a7 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:29.569 4076 3051053529224 MainThread DEBUG oslo_concurrency.processutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000028\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpje90iomm" returned: 0 in 0.360s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:30.019 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Setting boot order for instance: instance-00000028: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:30.078 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:30.114 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '59663825-7BAF-48E2-9B41-73FBFF1E47F5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15547), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:30.251 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:30.251 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:30.251 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:30.251 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:30.270 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '59663825-7BAF-48E2-9B41-73FBFF1E47F5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113825), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:30.270 4076 3051053529224 MainThread DEBUG os_win._utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19167780876159668 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:30.276 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4287c071-9d2f-4f96-9379-200890157b54 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:30.276 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-4287c071-9d2f-4f96-9379-200890157b54 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:30.308 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-4287c071-9d2f-4f96-9379-200890157b54 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:30.345 4076 3051054569256 MainThread INFO nova.compute.manager [req-4287c071-9d2f-4f96-9379-200890157b54 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:30.426 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'C0B1938A-B26B-4653-ABF3-B4FE88D29507', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14666), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:30.560 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:30.560 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:30.560 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:30.560 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:30.560 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C0B1938A-B26B-4653-ABF3-B4FE88D29507', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112054), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:30.587 4076 3051054567848 MainThread DEBUG os_win._utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6986513137817383 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:30.591 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-9bc16510-1e46-4cbc-beb9-f4cbe197a817 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:30.591 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-9bc16510-1e46-4cbc-beb9-f4cbe197a817 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:30.629 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-9bc16510-1e46-4cbc-beb9-f4cbe197a817 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:30.663 4076 3051054569256 MainThread INFO nova.compute.manager [req-9bc16510-1e46-4cbc-beb9-f4cbe197a817 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:30.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:30.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:30.808 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:30.808 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:30.824 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:30.824 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:55:30.824 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:30.843 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:31.036 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-bb3dd65d-b6bc-4ea1-9fa2-7bbfa3ed81c5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Received event network-vif-plugged-cd6ab33b-3f48-4910-aac6-7d9f710bee54 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:31.036 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-bb3dd65d-b6bc-4ea1-9fa2-7bbfa3ed81c5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:31.036 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-bb3dd65d-b6bc-4ea1-9fa2-7bbfa3ed81c5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:31.036 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-bb3dd65d-b6bc-4ea1-9fa2-7bbfa3ed81c5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Processing event network-vif-plugged-cd6ab33b-3f48-4910-aac6-7d9f710bee54 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:31.075 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Using config drive for instance 2021-09-10 16:55:31.075 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2138MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:55:31.075 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 714fea12-52ae-4277-8f3d-b0db2b2ee548 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5bd3ad21-3959-40c2-8698-b816b7176efc actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f0c24368-9e06-4126-bbb0-d52db1f2141b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.292 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:55:31.303 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 16 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:55:31.303 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=16896MB phys_disk=79GB used_disk=15GB total_vcpus=6 used_vcpus=16 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:55:31.311 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '80A84E69-24C8-426C-B092-218670CE2FA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=33532), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:31.339 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:31.349 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:31.440 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:31.440 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:31.440 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:31.440 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:31.453 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '80A84E69-24C8-426C-B092-218670CE2FA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112993), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:31.453 4076 3051053529224 MainThread DEBUG os_win._utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3746616840362549 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:31.453 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-4d813333-7188-469f-913b-d443f2c80d6d - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:31.453 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-4d813333-7188-469f-913b-d443f2c80d6d - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:31.513 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-4d813333-7188-469f-913b-d443f2c80d6d - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:31.542 4076 3051092583000 MainThread INFO nova.compute.manager [req-4d813333-7188-469f-913b-d443f2c80d6d - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:31.573 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:31.589 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:31.589 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:55:31.589 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.515s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:31.616 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'F1D2566C-A04A-4AE1-A4E2-FBDE92EE7BA8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:31.733 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Creating config drive at C:\openstack\instances\instance-00000029\configdrive.iso 2021-09-10 16:55:31.769 4076 3051053530104 MainThread DEBUG oslo_concurrency.processutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000029\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpujk_k1k0 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:31.918 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F1D2566C-A04A-4AE1-A4E2-FBDE92EE7BA8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:31.918 4076 3051054567848 MainThread DEBUG os_win._utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.029468297958374 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:31.918 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:31.918 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:31.918 4076 3051053529048 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:31.918 4076 3051053529048 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:31.955 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-d5103457-2f25-47fe-b418-2b0dae7d0740 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:31.955 4076 3051053529048 MainThread DEBUG nova.virt.hyperv.vmops [req-d5103457-2f25-47fe-b418-2b0dae7d0740 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:31.997 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-d5103457-2f25-47fe-b418-2b0dae7d0740 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:32.032 4076 3051053529048 MainThread INFO nova.compute.manager [req-d5103457-2f25-47fe-b418-2b0dae7d0740 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:32.075 4076 3051053530104 MainThread DEBUG oslo_concurrency.processutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000029\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpujk_k1k0" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:32.464 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '38B20403-9E7B-4049-A5D4-323495ABFA6A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15367), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:32.513 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Setting boot order for instance: instance-00000029: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:32.560 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:32.591 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '79F643E5-E1D6-4E6A-8F4D-65609C03CA8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17307), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:32.609 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:32.609 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:32.609 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:32.609 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:32.609 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:32.609 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '38B20403-9E7B-4049-A5D4-323495ABFA6A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:32.609 4076 3051053529224 MainThread DEBUG os_win._utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.53116512298584 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:32.656 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-5b1f8a6a-2f0e-49f4-9a44-6d65266f706a - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:32.656 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-5b1f8a6a-2f0e-49f4-9a44-6d65266f706a - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:32.714 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-5b1f8a6a-2f0e-49f4-9a44-6d65266f706a - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:32.729 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:32.729 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:32.729 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:32.729 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:32.745 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '79F643E5-E1D6-4E6A-8F4D-65609C03CA8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117060), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:32.745 4076 3051053530104 MainThread DEBUG os_win._utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18466734886169434 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:32.745 4076 3051054569256 MainThread INFO nova.compute.manager [req-5b1f8a6a-2f0e-49f4-9a44-6d65266f706a - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:32.776 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a73e4ebb-775a-47b3-a6e1-c3c1a4fe1717 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:32.776 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-a73e4ebb-775a-47b3-a6e1-c3c1a4fe1717 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:32.793 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:55:32.793 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:55:32.823 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a73e4ebb-775a-47b3-a6e1-c3c1a4fe1717 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:32.864 4076 3051092583000 MainThread INFO nova.compute.manager [req-a73e4ebb-775a-47b3-a6e1-c3c1a4fe1717 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:32.950 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'D414441F-9634-4A28-9D39-C0D8255FB118', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15745), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:33.083 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:33.083 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:33.095 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:33.095 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:33.095 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D414441F-9634-4A28-9D39-C0D8255FB118', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113665), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:33.095 4076 3051054567848 MainThread DEBUG os_win._utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.206243515014648 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:33.129 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-2fa8d9bb-6586-407d-8de0-ed84a2fe9a63 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:33.129 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-2fa8d9bb-6586-407d-8de0-ed84a2fe9a63 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:33.170 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-2fa8d9bb-6586-407d-8de0-ed84a2fe9a63 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:33.198 4076 3051092583000 MainThread INFO nova.compute.manager [req-2fa8d9bb-6586-407d-8de0-ed84a2fe9a63 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:33.621 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': 'AE87D65D-6657-477B-A507-8731D20766F8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15701), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:33.746 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:33.746 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:33.746 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:33.746 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:33.767 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AE87D65D-6657-477B-A507-8731D20766F8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110232), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:33.767 4076 3051053529224 MainThread DEBUG os_win._utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.688636064529419 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:33.788 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-842c8628-11a4-494b-915b-07d4b087a098 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Received event network-vif-plugged-747c7e5c-4098-4229-aaa0-b81088d50f7f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:33.788 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-842c8628-11a4-494b-915b-07d4b087a098 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:33.788 4076 3051054569256 MainThread DEBUG oslo_concurrency.lockutils [req-842c8628-11a4-494b-915b-07d4b087a098 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:33.788 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-842c8628-11a4-494b-915b-07d4b087a098 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Processing event network-vif-plugged-747c7e5c-4098-4229-aaa0-b81088d50f7f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:33.788 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'BCF74E34-D195-4EB4-B3A4-73524461C048', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14404), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:33.826 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-dd230d74-1ba4-45a1-b567-06da37aa72d8 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:33.826 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-dd230d74-1ba4-45a1-b567-06da37aa72d8 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:33.843 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Using config drive for instance 2021-09-10 16:55:33.872 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-dd230d74-1ba4-45a1-b567-06da37aa72d8 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:33.911 4076 3051092583000 MainThread INFO nova.compute.manager [req-dd230d74-1ba4-45a1-b567-06da37aa72d8 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:33.920 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:33.920 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:33.920 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:33.920 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:33.934 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-42257276-4266-4c92-a3ba-92501ca68ec2 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:33.934 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-42257276-4266-4c92-a3ba-92501ca68ec2 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:33.949 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BCF74E34-D195-4EB4-B3A4-73524461C048', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112872), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:33.949 4076 3051053530104 MainThread DEBUG os_win._utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3888370990753174 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:33.989 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-42257276-4266-4c92-a3ba-92501ca68ec2 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:34.012 4076 3051092583000 MainThread INFO nova.compute.manager [req-42257276-4266-4c92-a3ba-92501ca68ec2 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:34.106 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'EC847F58-DC40-4785-8D37-156B608D814A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15731), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:34.150 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Creating config drive at C:\openstack\instances\instance-0000002a\configdrive.iso 2021-09-10 16:55:34.187 4076 3051054571192 MainThread DEBUG oslo_concurrency.processutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpazjsa7b7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:34.354 4076 3051053529048 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:34.354 4076 3051053529048 MainThread INFO nova.compute.manager [-] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] VM Stopped (Lifecycle Event) 2021-09-10 16:55:34.354 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:34.354 4076 3051053530456 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:34.354 4076 3051054567848 MainThread ERROR os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000027\' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000027\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)instance-00000027C834F978-F6EC-4E6C-8E2D-608B1F269EC5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EC847F58-DC40-4785-8D37-156B608D814A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113953), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\nNot enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)\n\n'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5)", 'ErrorSummaryDescription': "'instance-00000027' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000027.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:34.354 4076 3051054567848 MainThread ERROR nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Failed to change vm state of instance-00000027 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. 2021-09-10 16:55:34.354 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Got request to destroy instance 2021-09-10 16:55:34.390 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-7f514608-cc2c-4a0e-9dbe-f7881449a220 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:34.390 4076 3051053529048 MainThread DEBUG nova.virt.hyperv.vmops [req-7f514608-cc2c-4a0e-9dbe-f7881449a220 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:34.442 4076 3051053529048 MainThread DEBUG nova.compute.manager [req-7f514608-cc2c-4a0e-9dbe-f7881449a220 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:34.474 4076 3051053529048 MainThread INFO nova.compute.manager [req-7f514608-cc2c-4a0e-9dbe-f7881449a220 - - - - -] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:34.505 4076 3051054571192 MainThread DEBUG oslo_concurrency.processutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpazjsa7b7" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:34.776 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '178F17A5-6127-4D07-A0B2-CA6CFE9A41BD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15162), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:34.940 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '178F17A5-6127-4D07-A0B2-CA6CFE9A41BD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119821), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:34.940 4076 3051053529224 MainThread DEBUG os_win._utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.862317800521851 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:34.940 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Setting boot order for instance: instance-0000002a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:34.940 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:34.940 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:34.940 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:34.940 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:34.969 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-e6c2f3e5-a149-4f40-96dd-ee28d91d8304 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:34.969 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-e6c2f3e5-a149-4f40-96dd-ee28d91d8304 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:34.996 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'B242D5A0-B070-415B-987A-98650C46C070', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20049), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:34.996 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:35.027 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-e6c2f3e5-a149-4f40-96dd-ee28d91d8304 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:35.027 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '025CD48C-5CD2-4860-B3C2-1E03771329D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18239), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:35.052 4076 3051054569256 MainThread INFO nova.compute.manager [req-e6c2f3e5-a149-4f40-96dd-ee28d91d8304 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:35.124 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:35.124 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:35.124 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:35.124 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:35.124 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B242D5A0-B070-415B-987A-98650C46C070', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114263), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:35.124 4076 3051053530104 MainThread DEBUG os_win._utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.563934326171875 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:35.144 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-56ed4fa4-1e0d-457b-a7b0-63429cbb5453 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:35.144 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-56ed4fa4-1e0d-457b-a7b0-63429cbb5453 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:35.192 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-56ed4fa4-1e0d-457b-a7b0-63429cbb5453 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:35.218 4076 3051054569256 MainThread INFO nova.compute.manager [req-56ed4fa4-1e0d-457b-a7b0-63429cbb5453 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:35.243 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:35.243 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:35.243 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:35.243 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:35.277 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-52dca17f-cc37-4218-b523-9b199758739e - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:35.277 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-52dca17f-cc37-4218-b523-9b199758739e - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:35.305 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '025CD48C-5CD2-4860-B3C2-1E03771329D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=188082), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:35.305 4076 3051054571192 MainThread DEBUG os_win._utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.3087034225463867 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:35.316 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-52dca17f-cc37-4218-b523-9b199758739e - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:35.339 4076 3051054569256 MainThread INFO nova.compute.manager [req-52dca17f-cc37-4218-b523-9b199758739e - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:35.961 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '641EC644-8A27-4236-A44C-6F791735FE16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:36.089 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.089 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.089 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:36.089 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] VM Stopped (Lifecycle Event) 2021-09-10 16:55:36.123 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-af67c936-39dc-4ba1-9cbe-3bf686a32258 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:36.123 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-af67c936-39dc-4ba1-9cbe-3bf686a32258 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:36.123 4076 3051053529224 MainThread ERROR os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000028\' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000028\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)instance-00000028C17232D7-F9BC-4B46-AD98-75DAF602F0E1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '641EC644-8A27-4236-A44C-6F791735FE16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112621), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\nNot enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)\n\n'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1)", 'ErrorSummaryDescription': "'instance-00000028' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000028.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:36.123 4076 3051053529224 MainThread ERROR nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Failed to change vm state of instance-00000028 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. 2021-09-10 16:55:36.123 4076 3051053529224 MainThread INFO nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Got request to destroy instance 2021-09-10 16:55:36.173 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-af67c936-39dc-4ba1-9cbe-3bf686a32258 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:36.173 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'D9D10795-FA34-47B7-A313-6FCD9F09EF12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16473), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:36.199 4076 3051054569256 MainThread INFO nova.compute.manager [req-af67c936-39dc-4ba1-9cbe-3bf686a32258 - - - - -] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:36.319 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.319 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.319 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:36.319 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:36.319 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D9D10795-FA34-47B7-A313-6FCD9F09EF12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=126909), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:36.319 4076 3051053530104 MainThread DEBUG os_win._utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7585887908935547 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:36.319 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:55:36.319 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.319 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.339 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-dfafc8ce-f307-44e2-87d6-bc9c37f86be6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:36.339 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-dfafc8ce-f307-44e2-87d6-bc9c37f86be6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:36.339 4076 3051054567848 MainThread DEBUG os_win.utils.compute.vmutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed vm state of instance-00000027 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:55:36.339 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Successfully changed state of VM instance-00000027 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:55:36.358 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '3A550DAC-154B-4228-8F1B-9542FCADFDDE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17636), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:36.391 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-dfafc8ce-f307-44e2-87d6-bc9c37f86be6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:36.403 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Waiting for WMI job: {'InstanceID': 'D3366749-10B9-4A70-A5F0-68C1ED905146', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18421), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:36.424 4076 3051054569256 MainThread INFO nova.compute.manager [req-dfafc8ce-f307-44e2-87d6-bc9c37f86be6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:36.512 4076 3051054569256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:36.512 4076 3051054569256 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:36.512 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.512 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.512 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3A550DAC-154B-4228-8F1B-9542FCADFDDE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=132430), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:36.512 4076 3051054571192 MainThread DEBUG os_win._utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.5157136917114258 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:36.528 4076 3051054567848 MainThread DEBUG os_win.utils.jobutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] WMI job succeeded: {'InstanceID': 'D3366749-10B9-4A70-A5F0-68C1ED905146', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=34035), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:55:36.528 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.528 4076 3051054567848 MainThread DEBUG os_win.utils.pathutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Removing directory: C:\openstack\instances\instance-00000027 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:55:36.528 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.528 4076 3051054567848 MainThread ERROR nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Traceback (most recent call last): 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(*args, **kwargs) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] admin_password, network_info, block_device_info) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(self, *args, **kwds) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] should_plug_vifs=False) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'req_state': req_state}) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] time.sleep(sleep_time) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return f(*args, **kwargs) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return self._wait_for_job(job_path) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] error_desc=err_desc) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During handling of the above exception, another exception occurred: 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Traceback (most recent call last): 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] yield resources 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] accel_info=accel_info) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(*args, **kwargs) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] admin_password, network_info, block_device_info) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(self, *args, **kwds) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] should_plug_vifs=False) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'req_state': req_state}) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] time.sleep(sleep_time) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return f(*args, **kwargs) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return self._wait_for_job(job_path) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] error_desc=err_desc) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:36.528 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:36.528 4076 3051054567848 MainThread INFO nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Terminating instance 2021-09-10 16:55:36.528 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:55:36.528 4076 3051054567848 MainThread INFO nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Got request to destroy instance 2021-09-10 16:55:36.544 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-c5d4f991-7093-4849-9fd5-eb29703a9304 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:36.544 4076 3051054569256 MainThread DEBUG nova.virt.hyperv.vmops [req-c5d4f991-7093-4849-9fd5-eb29703a9304 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:36.544 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:55:36.544 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8', 'target_lun': 0, 'volume_id': '52401085-77e7-4280-b78c-bad720b115d8', 'auth_method': 'CHAP', 'auth_username': 'Kn97TpVi2pyUkEmZbVJX', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:55:36.544 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:55:36.592 4076 3051054569256 MainThread DEBUG nova.compute.manager [req-c5d4f991-7093-4849-9fd5-eb29703a9304 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:36.605 4076 3051054569256 MainThread INFO nova.compute.manager [req-c5d4f991-7093-4849-9fd5-eb29703a9304 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:36.651 4076 3051054567848 MainThread DEBUG os_win.utils.storage.diskutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:55:36.651 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 16:55:36.715 4076 3051054567848 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-52401085-77e7-4280-b78c-bad720b115d8 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 16:55:36.715 4076 3051054567848 MainThread DEBUG os_brick.initiator.windows.iscsi [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] <== disconnect_volume: return (171ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:55:36.740 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:36.740 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "instance-00000027" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:36.740 4076 3051054567848 MainThread INFO nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Took 0.22 seconds to destroy the instance on the hypervisor. 2021-09-10 16:55:37.347 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'A94A4314-DC0A-43DC-AA0B-A16918C9E4A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15332), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:37.484 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:37.484 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:37.500 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:37.500 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:37.500 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A94A4314-DC0A-43DC-AA0B-A16918C9E4A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114097), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:37.500 4076 3051053530104 MainThread DEBUG os_win._utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.939236640930176 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:37.514 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-953fd95f-a07c-4eeb-bd2b-0bef0a738d62 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:37.514 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-953fd95f-a07c-4eeb-bd2b-0bef0a738d62 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:37.545 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': 'A059F572-17DE-45D0-B10F-79B4DEF69943', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16739), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:37.563 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-953fd95f-a07c-4eeb-bd2b-0bef0a738d62 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:37.606 4076 3051092583000 MainThread INFO nova.compute.manager [req-953fd95f-a07c-4eeb-bd2b-0bef0a738d62 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:37.668 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:37.668 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:37.668 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:37.668 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:37.694 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A059F572-17DE-45D0-B10F-79B4DEF69943', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115505), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:37.694 4076 3051054571192 MainThread DEBUG os_win._utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6974451541900635 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:37.694 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-488badec-c2f7-480e-ba33-baa9d6c902fb - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:37.694 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-488badec-c2f7-480e-ba33-baa9d6c902fb - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:37.739 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-488badec-c2f7-480e-ba33-baa9d6c902fb - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:37.809 4076 3051092583000 MainThread INFO nova.compute.manager [req-488badec-c2f7-480e-ba33-baa9d6c902fb - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:38.053 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:55:38.053 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.053 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.073 4076 3051053529224 MainThread DEBUG os_win.utils.compute.vmutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Successfully changed vm state of instance-00000028 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:55:38.078 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Successfully changed state of VM instance-00000028 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:55:38.120 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Waiting for WMI job: {'InstanceID': '72FE243E-8150-4B7B-8345-FA6681168C01', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15878), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:38.238 4076 3051053529224 MainThread DEBUG os_win.utils.jobutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] WMI job succeeded: {'InstanceID': '72FE243E-8150-4B7B-8345-FA6681168C01', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30938), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:55:38.238 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.238 4076 3051053529224 MainThread DEBUG os_win.utils.pathutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Removing directory: C:\openstack\instances\instance-00000028 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:55:38.238 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.238 4076 3051053529224 MainThread ERROR nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Traceback (most recent call last): 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(*args, **kwargs) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] admin_password, network_info, block_device_info) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(self, *args, **kwds) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] should_plug_vifs=False) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'req_state': req_state}) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] time.sleep(sleep_time) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return f(*args, **kwargs) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return self._wait_for_job(job_path) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] error_desc=err_desc) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During handling of the above exception, another exception occurred: 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Traceback (most recent call last): 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] yield resources 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] accel_info=accel_info) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(*args, **kwargs) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] admin_password, network_info, block_device_info) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(self, *args, **kwds) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] should_plug_vifs=False) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'req_state': req_state}) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] time.sleep(sleep_time) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return f(*args, **kwargs) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return self._wait_for_job(job_path) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] error_desc=err_desc) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.238 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.254 4076 3051053529224 MainThread INFO nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Terminating instance 2021-09-10 16:55:38.254 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:55:38.254 4076 3051053529224 MainThread INFO nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Got request to destroy instance 2021-09-10 16:55:38.271 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:55:38.271 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.271 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "instance-00000028" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.271 4076 3051053529224 MainThread INFO nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:55:38.271 4076 3051053529224 MainThread DEBUG nova.compute.claims [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:55:38.271 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.417 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.hostops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:38.436 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.hostops [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:38.520 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'CAF395A6-F652-45ED-A107-EC7F751F5C63', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15831), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:38.648 4076 3051053529224 MainThread DEBUG nova.compute.provider_tree [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:38.661 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:38.661 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:38.672 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.672 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.scheduler.client.report [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.672 4076 3051053529224 MainThread ERROR nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Traceback (most recent call last): 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(*args, **kwargs) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] admin_password, network_info, block_device_info) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(self, *args, **kwds) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] should_plug_vifs=False) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'req_state': req_state}) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] time.sleep(sleep_time) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return f(*args, **kwargs) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return self._wait_for_job(job_path) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] error_desc=err_desc) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] During handling of the above exception, another exception occurred: 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Traceback (most recent call last): 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] accel_info=accel_info) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(*args, **kwargs) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] admin_password, network_info, block_device_info) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return function(self, *args, **kwds) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] should_plug_vifs=False) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'req_state': req_state}) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] time.sleep(sleep_time) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self.force_reraise() 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] raise self.value 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return f(*args, **kwargs) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] return self._wait_for_job(job_path) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] error_desc=err_desc) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. 2021-09-10 16:55:38.672 4076 3051053529224 MainThread TRACE nova.compute.manager [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.compute.utils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Build of instance 714fea12-52ae-4277-8f3d-b0db2b2ee548 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000028' failed to start. Not enough memory in the system to start the virtual machine instance-00000028. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000028' failed to start. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Not enough memory in the system to start the virtual machine instance-00000028 with ram size 1024 megabytes. (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) 'instance-00000028' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C17232D7-F9BC-4B46-AD98-75DAF602F0E1) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:55:38.672 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:55:38.687 4076 3051053529224 MainThread DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:38.687 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-59bf4898-34eb-40d9-a691-2166ef0eaed6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:38.687 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-59bf4898-34eb-40d9-a691-2166ef0eaed6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:38.687 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000029\' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000029\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)instance-000000296E2346E6-ECC0-4DAC-B543-7248F2CCC9C4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CAF395A6-F652-45ED-A107-EC7F751F5C63', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113420), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\nNot enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)\n\n'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4)", 'ErrorSummaryDescription': "'instance-00000029' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000029.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:38.687 4076 3051053530104 MainThread ERROR nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Failed to change vm state of instance-00000029 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. 2021-09-10 16:55:38.687 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Got request to destroy instance 2021-09-10 16:55:38.733 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '570A458D-A210-4EFB-BB7F-13C11D786D3C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14688), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:38.733 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-59bf4898-34eb-40d9-a691-2166ef0eaed6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:38.771 4076 3051092583000 MainThread INFO nova.compute.manager [req-59bf4898-34eb-40d9-a691-2166ef0eaed6 - - - - -] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:38.873 4076 3051111972936 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:38.873 4076 3051111972936 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:38.873 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:38.873 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:38.873 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '570A458D-A210-4EFB-BB7F-13C11D786D3C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113264), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:38.873 4076 3051054571192 MainThread DEBUG os_win._utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8766777515411377 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:38.887 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-75ca02ae-d5e2-41a5-ab88-8820928c5901 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:38.887 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-75ca02ae-d5e2-41a5-ab88-8820928c5901 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:38.936 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-75ca02ae-d5e2-41a5-ab88-8820928c5901 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:38.965 4076 3051092583000 MainThread INFO nova.compute.manager [req-75ca02ae-d5e2-41a5-ab88-8820928c5901 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:39.909 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '630DF62A-ABBA-4880-9DA9-55B650310BF4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14988), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:40.046 4076 3051053530984 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:40.046 4076 3051053530984 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:40.046 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:40.046 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:40.046 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '630DF62A-ABBA-4880-9DA9-55B650310BF4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115328), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:40.046 4076 3051054571192 MainThread DEBUG os_win._utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.049975156784058 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:40.046 4076 3051053529224 MainThread DEBUG nova.network.neutron [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:40.092 4076 3051053529224 MainThread INFO nova.compute.manager [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] [instance: 714fea12-52ae-4277-8f3d-b0db2b2ee548] Took 1.41 seconds to deallocate network for instance. 2021-09-10 16:55:40.092 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-566a0adc-652b-476a-96e4-986ec73db3ab - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:40.092 4076 3051053530984 MainThread DEBUG nova.virt.hyperv.vmops [req-566a0adc-652b-476a-96e4-986ec73db3ab - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:40.152 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-566a0adc-652b-476a-96e4-986ec73db3ab - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:40.185 4076 3051053530984 MainThread INFO nova.compute.manager [req-566a0adc-652b-476a-96e4-986ec73db3ab - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:40.357 4076 3051053529224 MainThread INFO nova.scheduler.client.report [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Deleted allocations for instance 714fea12-52ae-4277-8f3d-b0db2b2ee548 2021-09-10 16:55:40.402 4076 3051053529224 MainThread DEBUG oslo_concurrency.lockutils [req-4fd51071-05db-489b-9e17-194f1680c759 82eb2bcce085415e8b748990e15c58a4 4ad3c7e6a94d4d6ca45d20d801674ca6 - default default] Lock "714fea12-52ae-4277-8f3d-b0db2b2ee548" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:40.558 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:55:40.558 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:40.558 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:40.584 4076 3051053530104 MainThread DEBUG os_win.utils.compute.vmutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed vm state of instance-00000029 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:55:40.584 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed state of VM instance-00000029 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:55:40.636 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '502EF2D8-2306-460A-873F-21E589AF6115', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14491), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:40.749 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job succeeded: {'InstanceID': '502EF2D8-2306-460A-873F-21E589AF6115', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30203), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:55:40.749 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:40.749 4076 3051053530104 MainThread DEBUG os_win.utils.pathutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Removing directory: C:\openstack\instances\instance-00000029 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:55:40.749 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:40.749 4076 3051053530104 MainThread ERROR nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Traceback (most recent call last): 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(*args, **kwargs) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] admin_password, network_info, block_device_info) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(self, *args, **kwds) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] should_plug_vifs=False) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'req_state': req_state}) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] time.sleep(sleep_time) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return f(*args, **kwargs) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return self._wait_for_job(job_path) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] error_desc=err_desc) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During handling of the above exception, another exception occurred: 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Traceback (most recent call last): 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] yield resources 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] accel_info=accel_info) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(*args, **kwargs) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] admin_password, network_info, block_device_info) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(self, *args, **kwds) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] should_plug_vifs=False) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'req_state': req_state}) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] time.sleep(sleep_time) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return f(*args, **kwargs) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return self._wait_for_job(job_path) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] error_desc=err_desc) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:40.749 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:40.764 4076 3051053530104 MainThread INFO nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Terminating instance 2021-09-10 16:55:40.764 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:55:40.764 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Got request to destroy instance 2021-09-10 16:55:40.764 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:55:40.764 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:40.764 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-00000029" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:40.764 4076 3051053530104 MainThread INFO nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 16:55:40.786 4076 3051053530104 MainThread DEBUG nova.compute.claims [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:55:40.786 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:41.046 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:41.068 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': 'A5213AE3-AE1B-4168-ACC2-92A16B3CF2C9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15156), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:41.068 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:41.170 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:41.183 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:41.216 4076 3051054571192 MainThread ERROR os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002a\' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)instance-0000002a4E972847-2F53-44DC-818C-E00C93D2FD6B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A5213AE3-AE1B-4168-ACC2-92A16B3CF2C9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117049), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\nNot enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)\n\n'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B)", 'ErrorSummaryDescription': "'instance-0000002a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:41.216 4076 3051054571192 MainThread ERROR nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Failed to change vm state of instance-0000002a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. 2021-09-10 16:55:41.216 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Got request to destroy instance 2021-09-10 16:55:41.216 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:41.216 4076 3051053529048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:41.216 4076 3051053529224 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:41.216 4076 3051053529224 MainThread INFO nova.compute.manager [-] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] VM Stopped (Lifecycle Event) 2021-09-10 16:55:41.267 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-2d1ebeb2-8482-49c6-9c8c-b0e024694465 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:41.267 4076 3051053529224 MainThread DEBUG nova.virt.hyperv.vmops [req-2d1ebeb2-8482-49c6-9c8c-b0e024694465 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:41.311 4076 3051053530104 MainThread DEBUG nova.compute.provider_tree [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:41.311 4076 3051053529224 MainThread DEBUG nova.compute.manager [req-2d1ebeb2-8482-49c6-9c8c-b0e024694465 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:41.355 4076 3051053529224 MainThread INFO nova.compute.manager [req-2d1ebeb2-8482-49c6-9c8c-b0e024694465 - - - - -] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.scheduler.client.report [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:41.387 4076 3051053530104 MainThread ERROR nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Traceback (most recent call last): 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(*args, **kwargs) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] admin_password, network_info, block_device_info) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(self, *args, **kwds) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] should_plug_vifs=False) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'req_state': req_state}) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] time.sleep(sleep_time) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return f(*args, **kwargs) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return self._wait_for_job(job_path) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] error_desc=err_desc) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] During handling of the above exception, another exception occurred: 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Traceback (most recent call last): 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] accel_info=accel_info) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(*args, **kwargs) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] admin_password, network_info, block_device_info) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return function(self, *args, **kwds) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] should_plug_vifs=False) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'req_state': req_state}) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] time.sleep(sleep_time) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self.force_reraise() 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] raise self.value 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return f(*args, **kwargs) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] return self._wait_for_job(job_path) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] error_desc=err_desc) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. 2021-09-10 16:55:41.387 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.compute.utils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Build of instance f0c24368-9e06-4126-bbb0-d52db1f2141b was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000029' failed to start. Not enough memory in the system to start the virtual machine instance-00000029. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000029' failed to start. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Not enough memory in the system to start the virtual machine instance-00000029 with ram size 1024 megabytes. (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) 'instance-00000029' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6E2346E6-ECC0-4DAC-B543-7248F2CCC9C4) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:55:41.387 4076 3051053530104 MainThread DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:41.444 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:41.444 4076 3051053528872 MainThread DEBUG nova.virt.hardware [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:41.444 4076 3051053528872 MainThread INFO nova.compute.claims [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Claim successful on node n-h1-808474-1 2021-09-10 16:55:41.614 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:41.637 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:41.829 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:41.917 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:41.917 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:41.917 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:42.035 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:42.035 4076 3051053530984 GreenThread-30 DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:42.035 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:42.065 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:42.106 4076 3051112034552 MainThread DEBUG nova.compute.manager [req-c1df72c2-faeb-47b1-b21e-581ab849c235 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Received event network-vif-unplugged-1e4619a1-db47-424e-b019-df43c556540c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:42.106 4076 3051112034552 MainThread DEBUG oslo_concurrency.lockutils [req-c1df72c2-faeb-47b1-b21e-581ab849c235 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:42.106 4076 3051112034552 MainThread DEBUG oslo_concurrency.lockutils [req-c1df72c2-faeb-47b1-b21e-581ab849c235 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:42.106 4076 3051112034552 MainThread DEBUG nova.compute.manager [req-c1df72c2-faeb-47b1-b21e-581ab849c235 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] No waiting events found dispatching network-vif-unplugged-1e4619a1-db47-424e-b019-df43c556540c pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 16:55:42.106 4076 3051112034552 MainThread WARNING nova.compute.manager [req-c1df72c2-faeb-47b1-b21e-581ab849c235 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Received unexpected event network-vif-unplugged-1e4619a1-db47-424e-b019-df43c556540c for instance with vm_state building and task_state spawning. 2021-09-10 16:55:42.305 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:42.320 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Spawning new instance 2021-09-10 16:55:42.336 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:42.336 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:42.336 4076 3051053528872 MainThread DEBUG nova.block_device [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:42.336 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:42.336 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:42.345 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Creating directory: C:\openstack\instances\instance-0000002b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:55:42.345 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:55:42.374 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Resizing VHD C:\openstack\instances\instance-0000002b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:55:42.486 4076 3051053530984 GreenThread-30 DEBUG nova.policy [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bd6f0f3184dd48b29f26ac7bd9ea6657', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef05fccd42414dbd84e1f21b80c6b71e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:42.639 4076 3051054567848 MainThread INFO nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Took 5.89 seconds to detach 1 volumes for instance. 2021-09-10 16:55:42.639 4076 3051054567848 MainThread DEBUG nova.compute.claims [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:55:42.639 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:42.787 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:42.810 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.hostops [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:43.026 4076 3051054567848 MainThread DEBUG nova.compute.provider_tree [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:43.042 4076 3051054567848 MainThread DEBUG nova.scheduler.client.report [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.049 4076 3051054567848 MainThread ERROR nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Traceback (most recent call last): 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(*args, **kwargs) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(self, *args, **kwds) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] should_plug_vifs=False) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'req_state': req_state}) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] time.sleep(sleep_time) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return f(*args, **kwargs) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return self._wait_for_job(job_path) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] error_desc=err_desc) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] During handling of the above exception, another exception occurred: 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Traceback (most recent call last): 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] accel_info=accel_info) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(*args, **kwargs) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return function(self, *args, **kwds) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] should_plug_vifs=False) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'req_state': req_state}) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] time.sleep(sleep_time) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self.force_reraise() 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] raise self.value 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return f(*args, **kwargs) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] return self._wait_for_job(job_path) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] error_desc=err_desc) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. 2021-09-10 16:55:43.049 4076 3051054567848 MainThread TRACE nova.compute.manager [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.compute.utils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Build of instance 5bd3ad21-3959-40c2-8698-b816b7176efc was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000027' failed to start. Not enough memory in the system to start the virtual machine instance-00000027. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000027' failed to start. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Not enough memory in the system to start the virtual machine instance-00000027 with ram size 1024 megabytes. (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) 'instance-00000027' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C834F978-F6EC-4E6C-8E2D-608B1F269EC5) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:55:43.049 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:43.126 4076 3051053530104 MainThread DEBUG nova.network.neutron [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:43.157 4076 3051053530104 MainThread INFO nova.compute.manager [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: f0c24368-9e06-4126-bbb0-d52db1f2141b] Took 1.77 seconds to deallocate network for instance. 2021-09-10 16:55:43.168 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:55:43.168 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:43.168 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.186 4076 3051054571192 MainThread DEBUG os_win.utils.compute.vmutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Successfully changed vm state of instance-0000002a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:55:43.186 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Successfully changed state of VM instance-0000002a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:55:43.223 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Waiting for WMI job: {'InstanceID': '87388789-F2B9-447B-B411-07504FA59F1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14345), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:43.362 4076 3051054571192 MainThread DEBUG os_win.utils.jobutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] WMI job succeeded: {'InstanceID': '87388789-F2B9-447B-B411-07504FA59F1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28798), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:55:43.362 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:43.362 4076 3051054571192 MainThread DEBUG os_win.utils.pathutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Removing directory: C:\openstack\instances\instance-0000002a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:55:43.362 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.362 4076 3051054571192 MainThread ERROR nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Traceback (most recent call last): 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(*args, **kwargs) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(self, *args, **kwds) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] should_plug_vifs=False) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'req_state': req_state}) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] time.sleep(sleep_time) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return f(*args, **kwargs) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return self._wait_for_job(job_path) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] error_desc=err_desc) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During handling of the above exception, another exception occurred: 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Traceback (most recent call last): 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] yield resources 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] accel_info=accel_info) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(*args, **kwargs) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(self, *args, **kwds) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] should_plug_vifs=False) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'req_state': req_state}) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] time.sleep(sleep_time) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return f(*args, **kwargs) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return self._wait_for_job(job_path) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] error_desc=err_desc) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.362 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.387 4076 3051054571192 MainThread INFO nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Terminating instance 2021-09-10 16:55:43.387 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:55:43.387 4076 3051054571192 MainThread INFO nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Got request to destroy instance 2021-09-10 16:55:43.401 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:55:43.401 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:43.401 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "instance-0000002a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.401 4076 3051054571192 MainThread INFO nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:55:43.401 4076 3051054571192 MainThread DEBUG nova.compute.claims [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:55:43.401 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:43.418 4076 3051053530104 MainThread INFO nova.scheduler.client.report [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Deleted allocations for instance f0c24368-9e06-4126-bbb0-d52db1f2141b 2021-09-10 16:55:43.543 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-961b2393-019a-4ec8-a6cc-7b832359a54e d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "f0c24368-9e06-4126-bbb0-d52db1f2141b" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.635 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:43.647 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.hostops [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:43.865 4076 3051054571192 MainThread DEBUG nova.compute.provider_tree [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.scheduler.client.report [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:43.903 4076 3051054571192 MainThread ERROR nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Traceback (most recent call last): 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(*args, **kwargs) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(self, *args, **kwds) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] should_plug_vifs=False) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'req_state': req_state}) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] time.sleep(sleep_time) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return f(*args, **kwargs) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return self._wait_for_job(job_path) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] error_desc=err_desc) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] During handling of the above exception, another exception occurred: 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Traceback (most recent call last): 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] accel_info=accel_info) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(*args, **kwargs) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] admin_password, network_info, block_device_info) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return function(self, *args, **kwds) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] should_plug_vifs=False) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'req_state': req_state}) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] time.sleep(sleep_time) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self.force_reraise() 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] raise self.value 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return f(*args, **kwargs) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] return self._wait_for_job(job_path) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] error_desc=err_desc) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. 2021-09-10 16:55:43.903 4076 3051054571192 MainThread TRACE nova.compute.manager [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.compute.utils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Build of instance 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002a' failed to start. Not enough memory in the system to start the virtual machine instance-0000002a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002a' failed to start. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Not enough memory in the system to start the virtual machine instance-0000002a with ram size 1024 megabytes. (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) 'instance-0000002a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E972847-2F53-44DC-818C-E00C93D2FD6B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:55:43.903 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:55:45.604 4076 3051054567848 MainThread DEBUG nova.network.neutron [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:45.667 4076 3051054567848 MainThread INFO nova.compute.manager [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 5bd3ad21-3959-40c2-8698-b816b7176efc] Took 2.62 seconds to deallocate network for instance. 2021-09-10 16:55:45.838 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Successfully created port: b622ae60-30c5-4ac6-a32f-35e5c15112ca _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:46.016 4076 3051054567848 MainThread INFO nova.scheduler.client.report [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Deleted allocations for instance 5bd3ad21-3959-40c2-8698-b816b7176efc 2021-09-10 16:55:46.068 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-b078a706-54d4-4c9c-a1dd-fdbb7214826a beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "5bd3ad21-3959-40c2-8698-b816b7176efc" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 40.094s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:46.077 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:46.111 4076 3051054571192 MainThread INFO nova.compute.manager [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] [instance: 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b] Took 2.20 seconds to deallocate network for instance. 2021-09-10 16:55:46.436 4076 3051054571192 MainThread INFO nova.scheduler.client.report [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Deleted allocations for instance 6aac7cdb-fa5e-417b-bf36-fe5eb161a90b 2021-09-10 16:55:46.474 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-dc0b74dc-15b3-46ef-83ca-685570e8cce0 14929f631b514b1ba7de48ea6f7875b6 f12d8e551e4d4d599a26e3249f7560f1 - default default] Lock "6aac7cdb-fa5e-417b-bf36-fe5eb161a90b" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.141s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:48.019 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Successfully updated port: b622ae60-30c5-4ac6-a32f-35e5c15112ca _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:48.047 4076 3051053530984 GreenThread-30 DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Acquired lock "refresh_cache-27ece020-a06e-4d29-aedc-cf0d87bdc941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:48.047 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:48.499 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:49.764 4076 3051053530984 GreenThread-30 DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Updating instance_info_cache with network_info: [{"id": "b622ae60-30c5-4ac6-a32f-35e5c15112ca", "address": "fa:16:3e:81:ab:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.202", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb622ae60-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:49.796 4076 3051053530984 GreenThread-30 DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Releasing lock "refresh_cache-27ece020-a06e-4d29-aedc-cf0d87bdc941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:49.796 4076 3051053530984 GreenThread-30 DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Instance network_info: |[{"id": "b622ae60-30c5-4ac6-a32f-35e5c15112ca", "address": "fa:16:3e:81:ab:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.202", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb622ae60-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:49.796 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Preparing to wait for external event network-vif-plugged-b622ae60-30c5-4ac6-a32f-35e5c15112ca prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:49.796 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:49.796 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:49.796 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Creating VM instance-0000002b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:50.669 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:50.720 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Received event network-changed-b622ae60-30c5-4ac6-a32f-35e5c15112ca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:50.720 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Refreshing instance network info cache due to event network-changed-b622ae60-30c5-4ac6-a32f-35e5c15112ca. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:50.720 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-27ece020-a06e-4d29-aedc-cf0d87bdc941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:50.720 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Refreshing network info cache for port b622ae60-30c5-4ac6-a32f-35e5c15112ca _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:52.559 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Updated VIF entry in instance network info cache for port b622ae60-30c5-4ac6-a32f-35e5c15112ca. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:52.559 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Updating instance_info_cache with network_info: [{"id": "b622ae60-30c5-4ac6-a32f-35e5c15112ca", "address": "fa:16:3e:81:ab:4d", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.202", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb622ae60-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:52.590 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-a1f90dae-3e52-4b43-a197-deae7667eff7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-27ece020-a06e-4d29-aedc-cf0d87bdc941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:53.544 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:53.558 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:55:53.659 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:53.659 4076 3051053530104 MainThread DEBUG nova.virt.hardware [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:55:53.659 4076 3051053530104 MainThread INFO nova.compute.claims [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Claim successful on node n-h1-808474-1 2021-09-10 16:55:53.796 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:55:53.816 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:55:54.044 4076 3051053530104 MainThread DEBUG nova.compute.provider_tree [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:55:54.064 4076 3051053530104 MainThread DEBUG nova.scheduler.client.report [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:55:54.064 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:54.064 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:55:54.140 4076 3051053530104 MainThread DEBUG nova.compute.utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:55:54.140 4076 3051053530984 GreenThread-31 DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:55:54.140 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:55:54.172 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:55:54.334 4076 3051053530984 GreenThread-31 DEBUG nova.policy [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bc0a11d23457425dafaa2590cadb3647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '26fab39a080a41888182e794dd32878e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:55:54.346 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:55:54.360 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Spawning new instance 2021-09-10 16:55:54.371 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:54.371 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:54.371 4076 3051053530104 MainThread DEBUG nova.block_device [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:55:54.371 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:54.371 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:54.385 4076 3051053530104 MainThread DEBUG os_win.utils.pathutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Creating directory: C:\openstack\instances\instance-0000002c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:55:54.385 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:55:54.424 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Resizing VHD C:\openstack\instances\instance-0000002c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:55:55.474 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Successfully created port: 18242dea-319e-41a3-8405-a9ea13fcf63d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:55:55.940 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-182bf21c-f541-4c15-8df9-9a0f6dd5a00c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Received event network-vif-plugged-b622ae60-30c5-4ac6-a32f-35e5c15112ca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:55.940 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-182bf21c-f541-4c15-8df9-9a0f6dd5a00c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:55.940 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [req-182bf21c-f541-4c15-8df9-9a0f6dd5a00c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:55.940 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-182bf21c-f541-4c15-8df9-9a0f6dd5a00c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Processing event network-vif-plugged-b622ae60-30c5-4ac6-a32f-35e5c15112ca _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:55:55.958 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Using config drive for instance 2021-09-10 16:55:56.255 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Creating config drive at C:\openstack\instances\instance-0000002b\configdrive.iso 2021-09-10 16:55:56.292 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpvc_28trp execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:55:56.610 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpvc_28trp" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:55:56.704 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Successfully updated port: 18242dea-319e-41a3-8405-a9ea13fcf63d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:55:56.719 4076 3051053530984 GreenThread-31 DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Acquired lock "refresh_cache-f8bc590e-89bd-458c-bef5-74846b896168" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:56.719 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:55:56.803 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:55:56.811 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Received event network-changed-18242dea-319e-41a3-8405-a9ea13fcf63d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:55:56.811 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Refreshing instance network info cache due to event network-changed-18242dea-319e-41a3-8405-a9ea13fcf63d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:55:57.011 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Setting boot order for instance: instance-0000002b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:55:57.058 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:55:57.079 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '54DB3910-859E-4684-B6E9-F91D617C6CC6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15193), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:57.151 4076 3051053530984 GreenThread-31 DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Updating instance_info_cache with network_info: [{"id": "18242dea-319e-41a3-8405-a9ea13fcf63d", "address": "fa:16:3e:35:8e:e3", "network": {"id": "dd5d187a-141b-46b6-bbf6-8f83ea71d54f", "bridge": null, "label": "tempest-ServersTestJSON-175667746-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "26fab39a080a41888182e794dd32878e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap18242dea-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:57.184 4076 3051053530984 GreenThread-31 DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Releasing lock "refresh_cache-f8bc590e-89bd-458c-bef5-74846b896168" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:57.184 4076 3051053530984 GreenThread-31 DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Instance network_info: |[{"id": "18242dea-319e-41a3-8405-a9ea13fcf63d", "address": "fa:16:3e:35:8e:e3", "network": {"id": "dd5d187a-141b-46b6-bbf6-8f83ea71d54f", "bridge": null, "label": "tempest-ServersTestJSON-175667746-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "26fab39a080a41888182e794dd32878e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap18242dea-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:55:57.184 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f8bc590e-89bd-458c-bef5-74846b896168" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:55:57.184 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Refreshing network info cache for port 18242dea-319e-41a3-8405-a9ea13fcf63d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:55:57.184 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Preparing to wait for external event network-vif-plugged-18242dea-319e-41a3-8405-a9ea13fcf63d prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:55:57.184 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:57.184 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:57.184 4076 3051053530104 MainThread DEBUG os_win.utils.compute.vmutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Creating VM instance-0000002c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:55:57.230 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '54DB3910-859E-4684-B6E9-F91D617C6CC6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116190), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:57.230 4076 3051053528872 MainThread DEBUG os_win._utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17181134223937988 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:57.250 4076 3051112035608 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:57.250 4076 3051112035608 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:55:57.250 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:57.250 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:57.278 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-3305b1de-f5a5-40a8-a5fa-d76b685e75e3 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:57.278 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-3305b1de-f5a5-40a8-a5fa-d76b685e75e3 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:57.333 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-3305b1de-f5a5-40a8-a5fa-d76b685e75e3 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:57.365 4076 3051112035608 MainThread INFO nova.compute.manager [req-3305b1de-f5a5-40a8-a5fa-d76b685e75e3 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:57.831 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Updated VIF entry in instance network info cache for port 18242dea-319e-41a3-8405-a9ea13fcf63d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:55:57.831 4076 3051054571192 MainThread DEBUG nova.network.neutron [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Updating instance_info_cache with network_info: [{"id": "18242dea-319e-41a3-8405-a9ea13fcf63d", "address": "fa:16:3e:35:8e:e3", "network": {"id": "dd5d187a-141b-46b6-bbf6-8f83ea71d54f", "bridge": null, "label": "tempest-ServersTestJSON-175667746-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "26fab39a080a41888182e794dd32878e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap18242dea-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:55:57.854 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [req-0d768915-9a99-4736-badf-e94d595a759b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f8bc590e-89bd-458c-bef5-74846b896168" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:55:58.112 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:55:58.247 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '8FCBF927-B8F4-4B47-A022-59BBB97A644F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15439), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:58.390 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:58.390 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:58.390 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:58.390 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:55:58.390 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8FCBF927-B8F4-4B47-A022-59BBB97A644F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112567), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:58.390 4076 3051053528872 MainThread DEBUG os_win._utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3315720558166504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:58.408 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-4e5883a0-f9e9-4681-a58b-75aa24ed55e7 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:58.408 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-4e5883a0-f9e9-4681-a58b-75aa24ed55e7 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:58.463 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-4e5883a0-f9e9-4681-a58b-75aa24ed55e7 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:58.471 4076 3051092583000 MainThread INFO nova.compute.manager [req-4e5883a0-f9e9-4681-a58b-75aa24ed55e7 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:55:59.414 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '8479BDEE-AD26-497E-9F8D-66E109586A25', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14288), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:55:59.540 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:55:59.540 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:55:59.540 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:55:59.540 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:55:59.562 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8479BDEE-AD26-497E-9F8D-66E109586A25', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108675), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 55, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 55, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:55:59.562 4076 3051053528872 MainThread DEBUG os_win._utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.504023790359497 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:55:59.562 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a0a9961f-10a0-4840-b2dd-533e07f36c0b - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:55:59.562 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-a0a9961f-10a0-4840-b2dd-533e07f36c0b - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:55:59.611 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a0a9961f-10a0-4840-b2dd-533e07f36c0b - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:55:59.620 4076 3051092583000 MainThread INFO nova.compute.manager [req-a0a9961f-10a0-4840-b2dd-533e07f36c0b - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:00.574 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '1FDC16AB-2172-4E8C-9A3E-A9C068F470AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14683), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:00.698 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:00.698 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:56:00.698 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:00.698 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:00.729 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1FDC16AB-2172-4E8C-9A3E-A9C068F470AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110341), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:00.729 4076 3051053528872 MainThread DEBUG os_win._utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6707348823547363 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:00.729 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-ef5c0736-6b0b-48ec-8ad1-c0ab2b9e1853 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:00.729 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-ef5c0736-6b0b-48ec-8ad1-c0ab2b9e1853 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:00.772 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-ef5c0736-6b0b-48ec-8ad1-c0ab2b9e1853 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:00.814 4076 3051092583000 MainThread INFO nova.compute.manager [req-ef5c0736-6b0b-48ec-8ad1-c0ab2b9e1853 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:01.754 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '4B7AC20D-33EE-4444-A461-C51C4D359195', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15311), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:01.888 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:01.888 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:56:01.888 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:01.888 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:01.911 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4B7AC20D-33EE-4444-A461-C51C4D359195', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111148), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:01.911 4076 3051053528872 MainThread DEBUG os_win._utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.853403568267822 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:01.911 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-8a3fb74f-2788-4209-ad33-04d2f6ec7985 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:01.911 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-8a3fb74f-2788-4209-ad33-04d2f6ec7985 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:01.948 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-8a3fb74f-2788-4209-ad33-04d2f6ec7985 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:01.965 4076 3051092583000 MainThread INFO nova.compute.manager [req-8a3fb74f-2788-4209-ad33-04d2f6ec7985 - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:02.949 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '34389A4C-C138-4B86-8808-6B4F51D18FC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:03.058 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:03.058 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:03.058 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:03.058 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] VM Stopped (Lifecycle Event) 2021-09-10 16:56:03.088 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002b\' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)instance-0000002bFB16A1F8-80B1-445B-BF86-AD4686A47AAA%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '34389A4C-C138-4B86-8808-6B4F51D18FC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110362), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\nNot enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)\n\n'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA)", 'ErrorSummaryDescription': "'instance-0000002b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:03.088 4076 3051053528872 MainThread ERROR nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Failed to change vm state of instance-0000002b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. 2021-09-10 16:56:03.088 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Got request to destroy instance 2021-09-10 16:56:03.088 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-fc732126-b2a9-400b-853d-16f2b7fe0c7a - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:03.088 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-fc732126-b2a9-400b-853d-16f2b7fe0c7a - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:03.155 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-fc732126-b2a9-400b-853d-16f2b7fe0c7a - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:03.169 4076 3051092583000 MainThread INFO nova.compute.manager [req-fc732126-b2a9-400b-853d-16f2b7fe0c7a - - - - -] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:04.824 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-cd9b184c-3ea2-4e5d-a453-1420efc0711c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Received event network-vif-plugged-18242dea-319e-41a3-8405-a9ea13fcf63d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:04.824 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-cd9b184c-3ea2-4e5d-a453-1420efc0711c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:04.824 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-cd9b184c-3ea2-4e5d-a453-1420efc0711c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:04.824 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-cd9b184c-3ea2-4e5d-a453-1420efc0711c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Processing event network-vif-plugged-18242dea-319e-41a3-8405-a9ea13fcf63d _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:04.865 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Using config drive for instance 2021-09-10 16:56:04.912 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:04.912 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:04.912 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:04.912 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Successfully changed vm state of instance-0000002b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:04.912 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Successfully changed state of VM instance-0000002b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:04.974 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Creating config drive at C:\openstack\instances\instance-0000002c\configdrive.iso 2021-09-10 16:56:05.027 4076 3051053530104 MainThread DEBUG oslo_concurrency.processutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpev1qrioz execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:05.162 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Waiting for WMI job: {'InstanceID': '71FEBE1D-F23A-427E-BBD9-44EE13DF6F75', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17567), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:05.272 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] WMI job succeeded: {'InstanceID': '71FEBE1D-F23A-427E-BBD9-44EE13DF6F75', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31641), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:05.272 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:05.272 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Removing directory: C:\openstack\instances\instance-0000002b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:05.292 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:05.292 4076 3051053528872 MainThread ERROR nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Traceback (most recent call last): 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(*args, **kwargs) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] admin_password, network_info, block_device_info) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(self, *args, **kwds) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] should_plug_vifs=False) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'req_state': req_state}) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] time.sleep(sleep_time) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return f(*args, **kwargs) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return self._wait_for_job(job_path) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] error_desc=err_desc) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During handling of the above exception, another exception occurred: 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Traceback (most recent call last): 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] yield resources 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] accel_info=accel_info) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(*args, **kwargs) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] admin_password, network_info, block_device_info) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(self, *args, **kwds) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] should_plug_vifs=False) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'req_state': req_state}) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] time.sleep(sleep_time) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return f(*args, **kwargs) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return self._wait_for_job(job_path) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] error_desc=err_desc) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.292 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.292 4076 3051053528872 MainThread INFO nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Terminating instance 2021-09-10 16:56:05.292 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:05.292 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Got request to destroy instance 2021-09-10 16:56:05.307 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:05.307 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:05.307 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "instance-0000002b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:05.307 4076 3051053528872 MainThread INFO nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:05.307 4076 3051053528872 MainThread DEBUG nova.compute.claims [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:05.321 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:05.342 4076 3051053530104 MainThread DEBUG oslo_concurrency.processutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpev1qrioz" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:05.416 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:05.428 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:05.643 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:05.668 4076 3051053528872 MainThread ERROR nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Traceback (most recent call last): 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(*args, **kwargs) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] admin_password, network_info, block_device_info) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(self, *args, **kwds) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] should_plug_vifs=False) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'req_state': req_state}) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] time.sleep(sleep_time) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return f(*args, **kwargs) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return self._wait_for_job(job_path) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] error_desc=err_desc) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] During handling of the above exception, another exception occurred: 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Traceback (most recent call last): 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] accel_info=accel_info) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(*args, **kwargs) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] admin_password, network_info, block_device_info) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return function(self, *args, **kwds) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] should_plug_vifs=False) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'req_state': req_state}) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] time.sleep(sleep_time) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self.force_reraise() 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] raise self.value 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return f(*args, **kwargs) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] return self._wait_for_job(job_path) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] error_desc=err_desc) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. 2021-09-10 16:56:05.668 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Build of instance 27ece020-a06e-4d29-aedc-cf0d87bdc941 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002b' failed to start. Not enough memory in the system to start the virtual machine instance-0000002b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002b' failed to start. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Not enough memory in the system to start the virtual machine instance-0000002b with ram size 1024 megabytes. (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) 'instance-0000002b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FB16A1F8-80B1-445B-BF86-AD4686A47AAA) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:05.668 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:05.745 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Setting boot order for instance: instance-0000002c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:05.807 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:05.838 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': 'CFC48466-57E4-4539-AC7C-2D0655918AAC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14248), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:05.948 4076 3051112036136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:05.948 4076 3051112036136 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:05.979 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:05.979 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:05.979 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CFC48466-57E4-4539-AC7C-2D0655918AAC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113407), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:05.979 4076 3051053530104 MainThread DEBUG os_win._utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17194294929504395 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:05.994 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-aab1028f-70a5-4b42-adaa-5bee316f5b2d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:05.994 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-aab1028f-70a5-4b42-adaa-5bee316f5b2d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:06.027 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-aab1028f-70a5-4b42-adaa-5bee316f5b2d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:06.060 4076 3051112036136 MainThread INFO nova.compute.manager [req-aab1028f-70a5-4b42-adaa-5bee316f5b2d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:06.184 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:06.209 4076 3051053528872 MainThread INFO nova.compute.manager [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] [instance: 27ece020-a06e-4d29-aedc-cf0d87bdc941] Took 0.55 seconds to deallocate network for instance. 2021-09-10 16:56:06.447 4076 3051053528872 MainThread INFO nova.scheduler.client.report [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Deleted allocations for instance 27ece020-a06e-4d29-aedc-cf0d87bdc941 2021-09-10 16:56:06.465 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-807d339a-de30-4b6e-8e3f-0ba4528a600b bd6f0f3184dd48b29f26ac7bd9ea6657 ef05fccd42414dbd84e1f21b80c6b71e - default default] Lock "27ece020-a06e-4d29-aedc-cf0d87bdc941" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:07.010 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': 'D6A073C2-11A7-468C-B9DC-140BB2737FB5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15269), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:07.139 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:07.139 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:07.139 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:07.139 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:07.157 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D6A073C2-11A7-468C-B9DC-140BB2737FB5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110773), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:07.157 4076 3051053530104 MainThread DEBUG os_win._utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3499085903167725 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:07.157 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-bfa869ad-e3a1-43c5-8ed4-91619cd1f1ed - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:07.157 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-bfa869ad-e3a1-43c5-8ed4-91619cd1f1ed - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:07.213 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-bfa869ad-e3a1-43c5-8ed4-91619cd1f1ed - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:07.270 4076 3051053528872 MainThread INFO nova.compute.manager [req-bfa869ad-e3a1-43c5-8ed4-91619cd1f1ed - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:08.167 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': '821E778E-C721-46BB-B402-901E8DDDD701', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14596), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:08.318 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:08.318 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:08.318 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:08.318 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:08.318 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '821E778E-C721-46BB-B402-901E8DDDD701', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111351), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:08.318 4076 3051053530104 MainThread DEBUG os_win._utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.510831832885742 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:08.338 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-aaf20f2f-2918-4499-9799-cb43d26e9d8f - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:08.338 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-aaf20f2f-2918-4499-9799-cb43d26e9d8f - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:08.380 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-aaf20f2f-2918-4499-9799-cb43d26e9d8f - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:08.428 4076 3051092583000 MainThread INFO nova.compute.manager [req-aaf20f2f-2918-4499-9799-cb43d26e9d8f - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:09.365 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': '79016ED7-7293-44CE-80B4-4F5B8038842C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:09.479 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:09.479 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:09.479 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:09.479 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:09.516 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '79016ED7-7293-44CE-80B4-4F5B8038842C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110960), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:09.516 4076 3051053530104 MainThread DEBUG os_win._utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7087955474853516 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:09.529 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-bc237966-a15e-45f8-ad6e-52eb6e1a298d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:09.529 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-bc237966-a15e-45f8-ad6e-52eb6e1a298d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:09.568 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-bc237966-a15e-45f8-ad6e-52eb6e1a298d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:09.588 4076 3051092583000 MainThread INFO nova.compute.manager [req-bc237966-a15e-45f8-ad6e-52eb6e1a298d - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:10.537 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': '0D5B0194-E0F7-4A57-90E4-AF67912CA8C7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14663), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:10.663 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:10.663 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:10.663 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:10.679 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:10.679 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0D5B0194-E0F7-4A57-90E4-AF67912CA8C7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110787), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:10.679 4076 3051053530104 MainThread DEBUG os_win._utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8717944622039795 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:10.700 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-dcfb2d77-8d52-41f3-99cd-561a283aedb4 - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:10.700 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-dcfb2d77-8d52-41f3-99cd-561a283aedb4 - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:10.740 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-dcfb2d77-8d52-41f3-99cd-561a283aedb4 - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:10.771 4076 3051092583000 MainThread INFO nova.compute.manager [req-dcfb2d77-8d52-41f3-99cd-561a283aedb4 - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:11.701 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': '798182F5-1E59-4C10-8587-894FD7C4871F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15556), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:11.858 4076 3051053530104 MainThread ERROR os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002c\' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F149610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)instance-0000002c8F742E89-88B4-4407-87A8-3C54BE5F1496%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '798182F5-1E59-4C10-8587-894FD7C4871F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120127), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\nNot enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)\n\n'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496)", 'ErrorSummaryDescription': "'instance-0000002c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:11.858 4076 3051053530104 MainThread ERROR nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Failed to change vm state of instance-0000002c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. 2021-09-10 16:56:11.858 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Got request to destroy instance 2021-09-10 16:56:11.858 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:11.858 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: f8bc590e-89bd-458c-bef5-74846b896168] VM Stopped (Lifecycle Event) 2021-09-10 16:56:11.858 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:11.858 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:11.889 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-c0fdab52-269a-493f-b8db-05f34b33562b - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:11.889 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-c0fdab52-269a-493f-b8db-05f34b33562b - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:11.932 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-c0fdab52-269a-493f-b8db-05f34b33562b - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:12.011 4076 3051092583000 MainThread INFO nova.compute.manager [req-c0fdab52-269a-493f-b8db-05f34b33562b - - - - -] [instance: f8bc590e-89bd-458c-bef5-74846b896168] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:13.775 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:13.777 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:13.777 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:13.777 4076 3051053530104 MainThread DEBUG os_win.utils.compute.vmutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Successfully changed vm state of instance-0000002c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:13.777 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Successfully changed state of VM instance-0000002c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:13.832 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Waiting for WMI job: {'InstanceID': '13C28DA8-6D7C-49D8-BB44-F260C1CD96B6', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14256), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:13.979 4076 3051053530104 MainThread DEBUG os_win.utils.jobutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] WMI job succeeded: {'InstanceID': '13C28DA8-6D7C-49D8-BB44-F260C1CD96B6', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28955), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:13.979 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:13.979 4076 3051053530104 MainThread DEBUG os_win.utils.pathutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Removing directory: C:\openstack\instances\instance-0000002c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:13.979 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:13.979 4076 3051053530104 MainThread ERROR nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Traceback (most recent call last): 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(*args, **kwargs) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] admin_password, network_info, block_device_info) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(self, *args, **kwds) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] should_plug_vifs=False) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'req_state': req_state}) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] time.sleep(sleep_time) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return f(*args, **kwargs) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return self._wait_for_job(job_path) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] error_desc=err_desc) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] During handling of the above exception, another exception occurred: 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Traceback (most recent call last): 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] yield resources 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] accel_info=accel_info) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(*args, **kwargs) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] admin_password, network_info, block_device_info) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(self, *args, **kwds) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] should_plug_vifs=False) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'req_state': req_state}) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] time.sleep(sleep_time) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return f(*args, **kwargs) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return self._wait_for_job(job_path) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] error_desc=err_desc) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:13.979 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:13.979 4076 3051053530104 MainThread INFO nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Terminating instance 2021-09-10 16:56:13.979 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:13.979 4076 3051053530104 MainThread INFO nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Got request to destroy instance 2021-09-10 16:56:14.006 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:14.006 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:14.006 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "instance-0000002c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:14.006 4076 3051053530104 MainThread INFO nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:14.006 4076 3051053530104 MainThread DEBUG nova.compute.claims [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:14.006 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:14.185 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:14.185 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.hostops [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:14.386 4076 3051053530104 MainThread DEBUG nova.compute.provider_tree [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:14.428 4076 3051053530104 MainThread DEBUG nova.scheduler.client.report [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:14.428 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:14.428 4076 3051053530104 MainThread ERROR nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Traceback (most recent call last): 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(*args, **kwargs) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] admin_password, network_info, block_device_info) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(self, *args, **kwds) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] should_plug_vifs=False) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'req_state': req_state}) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] time.sleep(sleep_time) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return f(*args, **kwargs) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return self._wait_for_job(job_path) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] error_desc=err_desc) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] During handling of the above exception, another exception occurred: 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Traceback (most recent call last): 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] accel_info=accel_info) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(*args, **kwargs) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] admin_password, network_info, block_device_info) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return function(self, *args, **kwds) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] should_plug_vifs=False) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'req_state': req_state}) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] time.sleep(sleep_time) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self.force_reraise() 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] raise self.value 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return f(*args, **kwargs) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] return self._wait_for_job(job_path) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] error_desc=err_desc) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. 2021-09-10 16:56:14.428 4076 3051053530104 MainThread TRACE nova.compute.manager [instance: f8bc590e-89bd-458c-bef5-74846b896168] 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.compute.utils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Build of instance f8bc590e-89bd-458c-bef5-74846b896168 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002c' failed to start. Not enough memory in the system to start the virtual machine instance-0000002c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002c' failed to start. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Not enough memory in the system to start the virtual machine instance-0000002c with ram size 1024 megabytes. (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) 'instance-0000002c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8F742E89-88B4-4407-87A8-3C54BE5F1496) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:14.434 4076 3051053530104 MainThread DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:15.938 4076 3051053530104 MainThread DEBUG nova.network.neutron [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:15.997 4076 3051053530104 MainThread INFO nova.compute.manager [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] [instance: f8bc590e-89bd-458c-bef5-74846b896168] Took 1.56 seconds to deallocate network for instance. 2021-09-10 16:56:16.256 4076 3051053530104 MainThread INFO nova.scheduler.client.report [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Deleted allocations for instance f8bc590e-89bd-458c-bef5-74846b896168 2021-09-10 16:56:16.312 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [req-9fdce9d4-9a7e-4be2-81fb-f1d18be9b084 bc0a11d23457425dafaa2590cadb3647 26fab39a080a41888182e794dd32878e - default default] Lock "f8bc590e-89bd-458c-bef5-74846b896168" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 22.766s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:16.897 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:16.923 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:17.033 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:17.033 4076 3051112035784 MainThread DEBUG nova.virt.hardware [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:17.033 4076 3051112035784 MainThread INFO nova.compute.claims [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Claim successful on node n-h1-808474-1 2021-09-10 16:56:17.261 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.hostops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:17.261 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.hostops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:17.480 4076 3051112035784 MainThread DEBUG nova.compute.provider_tree [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:17.510 4076 3051112035784 MainThread DEBUG nova.scheduler.client.report [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:17.510 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:17.510 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:17.621 4076 3051112035784 MainThread DEBUG nova.compute.utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:17.621 4076 3051053530104 GreenThread-32 DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:17.621 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:17.671 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:17.865 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:17.880 4076 3051112035784 MainThread INFO nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Spawning new instance 2021-09-10 16:56:17.880 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:17.880 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:17.880 4076 3051112035784 MainThread DEBUG nova.block_device [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:17.899 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:17.899 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:17.904 4076 3051112035784 MainThread DEBUG os_win.utils.pathutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating directory: C:\openstack\instances\instance-0000002e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:17.904 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:17.916 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Resizing VHD C:\openstack\instances\instance-0000002e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:18.065 4076 3051053530104 GreenThread-32 DEBUG nova.policy [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd33d11497a91483d93fe0ed8f5c26603', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'efc5deaa89294889821f303976e2d776', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:20.333 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Successfully created port: 9da1a72b-cb8f-4194-adf3-099206d8ea5c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:22.198 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:22.198 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:22.497 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:22.497 4076 3051053528872 MainThread DEBUG nova.virt.hardware [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:22.497 4076 3051053528872 MainThread INFO nova.compute.claims [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Claim successful on node n-h1-808474-1 2021-09-10 16:56:22.721 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:22.740 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:22.931 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:22.949 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:22.949 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:22.949 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:23.010 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Successfully updated port: 9da1a72b-cb8f-4194-adf3-099206d8ea5c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:23.037 4076 3051053530104 GreenThread-32 DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Acquired lock "refresh_cache-7e8913a3-2544-4f11-9354-908856f92cec" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:23.037 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:23.098 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:23.098 4076 3051054567848 GreenThread-33 DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:23.098 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:23.130 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:23.431 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:23.431 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Spawning new instance 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG nova.block_device [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Creating directory: C:\openstack\instances\instance-0000002f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:23.450 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:23.450 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:23.481 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Resizing VHD C:\openstack\instances\instance-0000002f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:23.744 4076 3051054567848 GreenThread-33 DEBUG nova.policy [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '73ac9100b70e4fa1b6ad7751b223e5ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '778686f367c642f39349bb208103a3ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:24.224 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Received event network-changed-9da1a72b-cb8f-4194-adf3-099206d8ea5c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:24.241 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Refreshing instance network info cache due to event network-changed-9da1a72b-cb8f-4194-adf3-099206d8ea5c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:24.697 4076 3051053530104 GreenThread-32 DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Updating instance_info_cache with network_info: [{"id": "9da1a72b-cb8f-4194-adf3-099206d8ea5c", "address": "fa:16:3e:82:d7:f4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.165", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9da1a72b-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:24.721 4076 3051053530104 GreenThread-32 DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Releasing lock "refresh_cache-7e8913a3-2544-4f11-9354-908856f92cec" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:24.721 4076 3051053530104 GreenThread-32 DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Instance network_info: |[{"id": "9da1a72b-cb8f-4194-adf3-099206d8ea5c", "address": "fa:16:3e:82:d7:f4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.165", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9da1a72b-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:24.721 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Preparing to wait for external event network-vif-plugged-9da1a72b-cb8f-4194-adf3-099206d8ea5c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:24.721 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:24.721 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:24.721 4076 3051112035784 MainThread DEBUG os_win.utils.compute.vmutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Creating VM instance-0000002e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:24.721 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-7e8913a3-2544-4f11-9354-908856f92cec" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:24.721 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Refreshing network info cache for port 9da1a72b-cb8f-4194-adf3-099206d8ea5c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:24.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:24.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 16:56:25.575 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:25.886 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Successfully created port: fef6956f-47c3-4e24-bec9-cca1b72ee9fe _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:26.260 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Updated VIF entry in instance network info cache for port 9da1a72b-cb8f-4194-adf3-099206d8ea5c. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:26.260 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Updating instance_info_cache with network_info: [{"id": "9da1a72b-cb8f-4194-adf3-099206d8ea5c", "address": "fa:16:3e:82:d7:f4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.165", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9da1a72b-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:26.271 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-5d712741-7775-4bc9-bf0e-dd525d1bcb21 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-7e8913a3-2544-4f11-9354-908856f92cec" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:27.416 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Successfully created port: 340a8a65-2d59-45c8-9a7d-18033bd9bbe8 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:28.801 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:28.801 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Successfully updated port: fef6956f-47c3-4e24-bec9-cca1b72ee9fe _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:28.801 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:29.427 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Received event network-changed-fef6956f-47c3-4e24-bec9-cca1b72ee9fe external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:29.427 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Refreshing instance network info cache due to event network-changed-fef6956f-47c3-4e24-bec9-cca1b72ee9fe. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:29.427 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:29.427 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Refreshing network info cache for port fef6956f-47c3-4e24-bec9-cca1b72ee9fe _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:29.655 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:29.729 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:29.752 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:29.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:29.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:56:29.890 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:29.890 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:56:30.145 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:30.145 4076 3051112035608 MainThread DEBUG nova.virt.hardware [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:30.145 4076 3051112035608 MainThread INFO nova.compute.claims [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Claim successful on node n-h1-808474-1 2021-09-10 16:56:30.308 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:30.355 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-b9f04a12-0719-450b-933d-0147a4914475 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:30.355 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 16:56:30.404 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 16:56:30.404 4076 3051112035608 MainThread DEBUG nova.compute.provider_tree [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 16:56:30.447 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 16:56:30.494 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 16:56:30.494 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.hostops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:30.494 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Successfully updated port: 340a8a65-2d59-45c8-9a7d-18033bd9bbe8 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:30.494 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.hostops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:30.521 4076 3051054567848 GreenThread-33 DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Acquired lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:30.521 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:30.636 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:30.743 4076 3051112035608 MainThread DEBUG nova.compute.provider_tree [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:30.752 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:30.752 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:30.752 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:30.916 4076 3051112035608 MainThread DEBUG nova.compute.utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:30.916 4076 3051053530104 GreenThread-34 DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:30.916 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:30.947 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:31.150 4076 3051053530104 GreenThread-34 DEBUG nova.policy [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a6076248095044dcbd42424ad33c0395', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2113aff5a9ed47f5bdb3bdee01c77f67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:31.169 4076 3051112035608 MainThread INFO nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Spawning new instance 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG nova.block_device [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:31.169 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:31.198 4076 3051112035608 MainThread DEBUG os_win.utils.pathutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Creating directory: C:\openstack\instances\instance-00000030 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:31.198 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000030\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:31.213 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Resizing VHD C:\openstack\instances\instance-00000030\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:31.418 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:31.418 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:31.418 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:56:31.418 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:31.418 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:31.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:31.811 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:31.811 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:31.811 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 16:56:31.822 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 16:56:32.244 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-aa17abe0-042b-4206-b288-67c436891b1b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Received event network-vif-plugged-9da1a72b-cb8f-4194-adf3-099206d8ea5c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:32.244 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-aa17abe0-042b-4206-b288-67c436891b1b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:32.244 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-aa17abe0-042b-4206-b288-67c436891b1b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:32.259 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-aa17abe0-042b-4206-b288-67c436891b1b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Processing event network-vif-plugged-9da1a72b-cb8f-4194-adf3-099206d8ea5c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:32.287 4076 3051112035784 MainThread INFO nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Using config drive for instance 2021-09-10 16:56:32.322 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Received event network-changed-340a8a65-2d59-45c8-9a7d-18033bd9bbe8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:32.322 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Refreshing instance network info cache due to event network-changed-340a8a65-2d59-45c8-9a7d-18033bd9bbe8. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:32.566 4076 3051054567848 GreenThread-33 DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Updating instance_info_cache with network_info: [{"id": "fef6956f-47c3-4e24-bec9-cca1b72ee9fe", "address": "fa:16:3e:30:c0:ea", "network": {"id": "4a075669-fa85-4572-a624-6452ab88ff78", "bridge": null, "label": "tempest-ServersTestMultiNic-1116835284", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfef6956f-47", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "340a8a65-2d59-45c8-9a7d-18033bd9bbe8", "address": "fa:16:3e:58:56:c6", "network": {"id": "3d66e9cf-4851-479f-b109-f0549cb5725c", "bridge": null, "label": "tempest-ServersTestMultiNic-33965206", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.26", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap340a8a65-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:32.599 4076 3051054567848 GreenThread-33 DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Releasing lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:32.599 4076 3051054567848 GreenThread-33 DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Instance network_info: |[{"id": "fef6956f-47c3-4e24-bec9-cca1b72ee9fe", "address": "fa:16:3e:30:c0:ea", "network": {"id": "4a075669-fa85-4572-a624-6452ab88ff78", "bridge": null, "label": "tempest-ServersTestMultiNic-1116835284", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfef6956f-47", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "340a8a65-2d59-45c8-9a7d-18033bd9bbe8", "address": "fa:16:3e:58:56:c6", "network": {"id": "3d66e9cf-4851-479f-b109-f0549cb5725c", "bridge": null, "label": "tempest-ServersTestMultiNic-33965206", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.26", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap340a8a65-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Preparing to wait for external event network-vif-plugged-fef6956f-47c3-4e24-bec9-cca1b72ee9fe prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Preparing to wait for external event network-vif-plugged-340a8a65-2d59-45c8-9a7d-18033bd9bbe8 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:32.599 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Creating VM instance-0000002f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:32.599 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:32.599 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Refreshing network info cache for port 340a8a65-2d59-45c8-9a7d-18033bd9bbe8 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:32.806 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:32.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:32.833 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:32.833 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:32.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:56:32.833 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:32.864 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:32.878 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Successfully created port: 65475916-c478-427b-b1e3-86dd5da029e4 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:33.076 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2141MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:56:33.092 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:33.092 4076 3051112035784 MainThread INFO nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Creating config drive at C:\openstack\instances\instance-0000002e\configdrive.iso 2021-09-10 16:56:33.126 4076 3051112035784 MainThread DEBUG oslo_concurrency.processutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4f0alx2j execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:33.434 4076 3051112035784 MainThread DEBUG oslo_concurrency.processutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4f0alx2j" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a1386320-87c7-493d-a393-4e590b06faa0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a5380419-a846-48fc-891c-d88dc8215cfd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7e8913a3-2544-4f11-9354-908856f92cec actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 15 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:56:33.481 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=15872MB phys_disk=79GB used_disk=15GB total_vcpus=6 used_vcpus=15 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:56:33.526 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:33.526 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:33.622 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:33.684 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:33.769 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:33.784 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:33.784 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:56:33.784 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:33.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:33.901 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Setting boot order for instance: instance-0000002e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:33.948 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:33.985 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'C8BE0DFA-749C-4814-80C1-A18085B5764F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:34.090 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Updated VIF entry in instance network info cache for port 340a8a65-2d59-45c8-9a7d-18033bd9bbe8. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:34.090 4076 3051053530984 MainThread DEBUG nova.network.neutron [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Updating instance_info_cache with network_info: [{"id": "fef6956f-47c3-4e24-bec9-cca1b72ee9fe", "address": "fa:16:3e:30:c0:ea", "network": {"id": "4a075669-fa85-4572-a624-6452ab88ff78", "bridge": null, "label": "tempest-ServersTestMultiNic-1116835284", "subnets": [{"cidr": "19.80.0.0/24", "dns": [], "gateway": {"address": "19.80.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.80.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.80.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfef6956f-47", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "340a8a65-2d59-45c8-9a7d-18033bd9bbe8", "address": "fa:16:3e:58:56:c6", "network": {"id": "3d66e9cf-4851-479f-b109-f0549cb5725c", "bridge": null, "label": "tempest-ServersTestMultiNic-33965206", "subnets": [{"cidr": "19.86.0.0/24", "dns": [], "gateway": {"address": "19.86.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "19.86.0.26", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "19.86.0.2"}}], "meta": {"injected": false, "tenant_id": "778686f367c642f39349bb208103a3ca", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap340a8a65-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:34.106 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:34.106 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:34.121 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:34.121 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:34.121 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [req-8fb39168-2455-4127-b16d-2f6639ca2c71 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a5380419-a846-48fc-891c-d88dc8215cfd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:34.121 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C8BE0DFA-749C-4814-80C1-A18085B5764F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113187), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:34.121 4076 3051112035784 MainThread DEBUG os_win._utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1731116771697998 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:34.150 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-67b53ea9-6475-44c9-91be-f8b2ff77bb9b - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:34.150 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-67b53ea9-6475-44c9-91be-f8b2ff77bb9b - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:34.196 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-67b53ea9-6475-44c9-91be-f8b2ff77bb9b - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:34.216 4076 3051092583000 MainThread INFO nova.compute.manager [req-67b53ea9-6475-44c9-91be-f8b2ff77bb9b - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:34.447 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Successfully updated port: 65475916-c478-427b-b1e3-86dd5da029e4 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:34.466 4076 3051053530104 GreenThread-34 DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Acquired lock "refresh_cache-a1386320-87c7-493d-a393-4e590b06faa0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:34.466 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:34.466 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:34.466 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:34.489 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:34.489 4076 3051092583000 MainThread ERROR nova.compute.manager [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Traceback (most recent call last): 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] self._delete_instance(context, instance, bdms) 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] NameError: name 'instant' is not defined 2021-09-10 16:56:34.489 4076 3051092583000 MainThread TRACE nova.compute.manager [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] 2021-09-10 16:56:34.587 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:34.717 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:34.717 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:34.759 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.hostops [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:34.765 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.hostops [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:34.987 4076 3051092583000 MainThread DEBUG nova.compute.provider_tree [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:35.000 4076 3051092583000 MainThread DEBUG nova.scheduler.client.report [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:35.000 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.282s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:35.000 4076 3051092583000 MainThread INFO nova.compute.manager [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Successfully reverted task state from deleting on failure for instance. 2021-09-10 16:56:35.015 4076 3051092583000 MainThread ERROR oslo_messaging.rpc.server [req-158ee550-840b-41cc-a631-fe460c1cba2b beffe9ba8529426fbb27e549254bd679 91237ebda1b54d2dbc761e48fdd93516 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 16:56:35.015 4076 3051092583000 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 16:56:35.137 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '628F0F7A-C2FA-4F9F-8A44-803350F7A4DD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15160), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:35.151 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Received event network-changed-65475916-c478-427b-b1e3-86dd5da029e4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:35.151 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Refreshing instance network info cache due to event network-changed-65475916-c478-427b-b1e3-86dd5da029e4. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:35.268 4076 3051053530104 GreenThread-34 DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Updating instance_info_cache with network_info: [{"id": "65475916-c478-427b-b1e3-86dd5da029e4", "address": "fa:16:3e:d6:a7:e5", "network": {"id": "f10caebc-9a04-4a48-a217-b55195b3ed60", "bridge": null, "label": "tempest-InstanceActionsNegativeTestJSON-271879686-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "2113aff5a9ed47f5bdb3bdee01c77f67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap65475916-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:35.268 4076 3051053530984 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:35.268 4076 3051053530984 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:35.291 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:35.291 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:35.291 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '628F0F7A-C2FA-4F9F-8A44-803350F7A4DD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110634), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:35.291 4076 3051112035784 MainThread DEBUG os_win._utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.342836618423462 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:35.310 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-ced0589e-f1dd-4d1b-bd46-e38488bfbb11 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:35.310 4076 3051053530984 MainThread DEBUG nova.virt.hyperv.vmops [req-ced0589e-f1dd-4d1b-bd46-e38488bfbb11 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:35.310 4076 3051053530104 GreenThread-34 DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Releasing lock "refresh_cache-a1386320-87c7-493d-a393-4e590b06faa0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:35.310 4076 3051053530104 GreenThread-34 DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Instance network_info: |[{"id": "65475916-c478-427b-b1e3-86dd5da029e4", "address": "fa:16:3e:d6:a7:e5", "network": {"id": "f10caebc-9a04-4a48-a217-b55195b3ed60", "bridge": null, "label": "tempest-InstanceActionsNegativeTestJSON-271879686-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "2113aff5a9ed47f5bdb3bdee01c77f67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap65475916-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:35.310 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Preparing to wait for external event network-vif-plugged-65475916-c478-427b-b1e3-86dd5da029e4 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:35.310 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:35.310 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:35.310 4076 3051112035608 MainThread DEBUG os_win.utils.compute.vmutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Creating VM instance-00000030 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:35.310 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a1386320-87c7-493d-a393-4e590b06faa0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:35.310 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Refreshing network info cache for port 65475916-c478-427b-b1e3-86dd5da029e4 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:35.340 4076 3051053530984 MainThread DEBUG nova.compute.manager [req-ced0589e-f1dd-4d1b-bd46-e38488bfbb11 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:35.405 4076 3051053530984 MainThread INFO nova.compute.manager [req-ced0589e-f1dd-4d1b-bd46-e38488bfbb11 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:35.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:56:35.771 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:56:36.041 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Updated VIF entry in instance network info cache for port 65475916-c478-427b-b1e3-86dd5da029e4. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:36.041 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Updating instance_info_cache with network_info: [{"id": "65475916-c478-427b-b1e3-86dd5da029e4", "address": "fa:16:3e:d6:a7:e5", "network": {"id": "f10caebc-9a04-4a48-a217-b55195b3ed60", "bridge": null, "label": "tempest-InstanceActionsNegativeTestJSON-271879686-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "2113aff5a9ed47f5bdb3bdee01c77f67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap65475916-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:36.056 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-7b2b84a2-d9a0-4f2a-ac34-6bcb2216f83f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a1386320-87c7-493d-a393-4e590b06faa0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:36.234 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:36.312 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '113C7436-512B-4D6C-8F7B-B66D0326CC4E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14323), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:36.451 4076 3051053530104 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:36.451 4076 3051053530104 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:36.468 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:36.468 4076 3051053530984 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:36.468 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '113C7436-512B-4D6C-8F7B-B66D0326CC4E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114406), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:36.468 4076 3051112035784 MainThread DEBUG os_win._utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.519446611404419 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:36.489 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-15410f5d-7206-4983-bbca-07534885fba6 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:36.489 4076 3051053530104 MainThread DEBUG nova.virt.hyperv.vmops [req-15410f5d-7206-4983-bbca-07534885fba6 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:36.521 4076 3051053530104 MainThread DEBUG nova.compute.manager [req-15410f5d-7206-4983-bbca-07534885fba6 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:36.572 4076 3051053530104 MainThread INFO nova.compute.manager [req-15410f5d-7206-4983-bbca-07534885fba6 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:37.447 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:37.465 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:37.497 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'BD714A0B-2448-4C40-85A9-D01BCEE00A06', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20698), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:37.604 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:37.604 4076 3051112036488 MainThread DEBUG nova.virt.hardware [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:37.604 4076 3051112036488 MainThread INFO nova.compute.claims [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Claim successful on node n-h1-808474-1 2021-09-10 16:56:37.636 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:37.636 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:37.636 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:37.636 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:37.666 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BD714A0B-2448-4C40-85A9-D01BCEE00A06', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111116), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:37.666 4076 3051112035784 MainThread DEBUG os_win._utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7173783779144287 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:37.666 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-9bc84a10-1eaf-4e37-977d-6d2cd405e319 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:37.666 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-9bc84a10-1eaf-4e37-977d-6d2cd405e319 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:37.736 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-9bc84a10-1eaf-4e37-977d-6d2cd405e319 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:37.759 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.hostops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:37.762 4076 3051054571192 MainThread INFO nova.compute.manager [req-9bc84a10-1eaf-4e37-977d-6d2cd405e319 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:37.762 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.hostops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:38.000 4076 3051112036488 MainThread DEBUG nova.compute.provider_tree [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:38.033 4076 3051112036488 MainThread DEBUG nova.scheduler.client.report [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:38.033 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:38.033 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:38.130 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:38.130 4076 3051054571192 GreenThread-35 DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:38.130 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:38.156 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-84bc3be6-a7d6-456d-96ca-71d0c3a0ae9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Received event network-vif-deleted-cedaf97e-b489-4b71-abd1-950cba1f0106 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:38.369 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:38.389 4076 3051112036488 MainThread INFO nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Spawning new instance 2021-09-10 16:56:38.389 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:38.389 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:38.389 4076 3051112036488 MainThread DEBUG nova.block_device [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:38.389 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:38.389 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:38.409 4076 3051112036488 MainThread DEBUG os_win.utils.pathutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Creating directory: C:\openstack\instances\instance-0000002d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:38.409 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000002d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:38.417 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Resizing VHD C:\openstack\instances\instance-0000002d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:38.717 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '9D5C5550-3299-4215-A5F4-173E4365F502', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15438), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:38.797 4076 3051054571192 GreenThread-35 DEBUG nova.policy [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9bc7e248f98441caa76226a9e8d21746', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e122cd09445d480a826d8bd883b15cad', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:38.862 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9D5C5550-3299-4215-A5F4-173E4365F502', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124482), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:38.862 4076 3051112035784 MainThread DEBUG os_win._utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.914019346237183 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:38.862 4076 3051112034376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:38.862 4076 3051112034376 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:38.862 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:38.862 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:38.885 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-fdd4ef6c-c0fd-4b8d-8374-7dde01ae7495 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:38.885 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-fdd4ef6c-c0fd-4b8d-8374-7dde01ae7495 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:38.944 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-fdd4ef6c-c0fd-4b8d-8374-7dde01ae7495 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:39.057 4076 3051112034376 MainThread INFO nova.compute.manager [req-fdd4ef6c-c0fd-4b8d-8374-7dde01ae7495 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:39.836 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Successfully created port: 7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:39.892 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': '55E7A178-1F41-45B2-AFCF-33E4DA625291', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14726), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:40.010 4076 3051112034376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:40.010 4076 3051112034376 MainThread INFO nova.compute.manager [-] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] VM Stopped (Lifecycle Event) 2021-09-10 16:56:40.036 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:40.036 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:40.036 4076 3051112035784 MainThread ERROR os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002e\' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)instance-0000002e1EE466B8-D3BB-4736-9900-FA2EF09DDF1F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '55E7A178-1F41-45B2-AFCF-33E4DA625291', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112651), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\nNot enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)\n\n'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F)", 'ErrorSummaryDescription': "'instance-0000002e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:40.041 4076 3051112035784 MainThread ERROR nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Failed to change vm state of instance-0000002e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. 2021-09-10 16:56:40.041 4076 3051112035784 MainThread INFO nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Got request to destroy instance 2021-09-10 16:56:40.076 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-5d7b89c9-e249-4c21-bf1d-7e066ab57e51 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:40.076 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-5d7b89c9-e249-4c21-bf1d-7e066ab57e51 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:40.136 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-5d7b89c9-e249-4c21-bf1d-7e066ab57e51 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:40.167 4076 3051112034376 MainThread INFO nova.compute.manager [req-5d7b89c9-e249-4c21-bf1d-7e066ab57e51 - - - - -] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:41.061 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Successfully updated port: 7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:41.091 4076 3051054571192 GreenThread-35 DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Acquired lock "refresh_cache-9d815a45-c30c-4ca8-a66d-6ef7fef6185d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:41.091 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:41.201 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:41.390 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Received event network-changed-7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:41.390 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Refreshing instance network info cache due to event network-changed-7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:41.921 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:41.921 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:41.921 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:41.934 4076 3051112035784 MainThread DEBUG os_win.utils.compute.vmutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed vm state of instance-0000002e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:41.934 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Successfully changed state of VM instance-0000002e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:41.974 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Waiting for WMI job: {'InstanceID': 'BAD44111-69B3-4D0A-9295-42B44A2A51BA', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15126), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:42.061 4076 3051112035432 MainThread DEBUG nova.compute.manager [req-260770db-13fc-41de-8ece-a38516338585 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Received event network-vif-plugged-fef6956f-47c3-4e24-bec9-cca1b72ee9fe external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:42.061 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [req-260770db-13fc-41de-8ece-a38516338585 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:42.061 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [req-260770db-13fc-41de-8ece-a38516338585 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:42.061 4076 3051112035432 MainThread DEBUG nova.compute.manager [req-260770db-13fc-41de-8ece-a38516338585 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Processing event network-vif-plugged-fef6956f-47c3-4e24-bec9-cca1b72ee9fe _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:42.101 4076 3051112035784 MainThread DEBUG os_win.utils.jobutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] WMI job succeeded: {'InstanceID': 'BAD44111-69B3-4D0A-9295-42B44A2A51BA', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29407), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:42.105 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:42.105 4076 3051112035784 MainThread DEBUG os_win.utils.pathutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Removing directory: C:\openstack\instances\instance-0000002e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:42.105 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:42.105 4076 3051112035784 MainThread ERROR nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Traceback (most recent call last): 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(*args, **kwargs) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] admin_password, network_info, block_device_info) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(self, *args, **kwds) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] should_plug_vifs=False) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'req_state': req_state}) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] time.sleep(sleep_time) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return f(*args, **kwargs) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return self._wait_for_job(job_path) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] error_desc=err_desc) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During handling of the above exception, another exception occurred: 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Traceback (most recent call last): 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] yield resources 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] accel_info=accel_info) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(*args, **kwargs) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] admin_password, network_info, block_device_info) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(self, *args, **kwds) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] should_plug_vifs=False) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'req_state': req_state}) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] time.sleep(sleep_time) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return f(*args, **kwargs) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return self._wait_for_job(job_path) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] error_desc=err_desc) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.105 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.105 4076 3051112035784 MainThread INFO nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Terminating instance 2021-09-10 16:56:42.105 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:42.105 4076 3051112035784 MainThread INFO nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Got request to destroy instance 2021-09-10 16:56:42.130 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.vmops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:42.130 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:42.130 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "instance-0000002e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:42.130 4076 3051112035784 MainThread INFO nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:42.130 4076 3051112035784 MainThread DEBUG nova.compute.claims [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:42.130 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:42.284 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.hostops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:42.318 4076 3051112035784 MainThread DEBUG nova.virt.hyperv.hostops [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:42.329 4076 3051054571192 GreenThread-35 DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Updating instance_info_cache with network_info: [{"id": "7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55", "address": "fa:16:3e:8b:69:01", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7b4ee10d-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:42.358 4076 3051054571192 GreenThread-35 DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Releasing lock "refresh_cache-9d815a45-c30c-4ca8-a66d-6ef7fef6185d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:42.358 4076 3051054571192 GreenThread-35 DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Instance network_info: |[{"id": "7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55", "address": "fa:16:3e:8b:69:01", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7b4ee10d-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:42.358 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9d815a45-c30c-4ca8-a66d-6ef7fef6185d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:42.358 4076 3051112034376 MainThread DEBUG nova.network.neutron [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Refreshing network info cache for port 7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:42.358 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Preparing to wait for external event network-vif-plugged-7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:42.358 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:42.358 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:42.358 4076 3051112036488 MainThread DEBUG os_win.utils.compute.vmutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Creating VM instance-0000002d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:42.521 4076 3051112035784 MainThread DEBUG nova.compute.provider_tree [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:42.540 4076 3051112035784 MainThread DEBUG nova.scheduler.client.report [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:42.540 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:42.540 4076 3051112035784 MainThread ERROR nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Traceback (most recent call last): 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(*args, **kwargs) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] admin_password, network_info, block_device_info) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(self, *args, **kwds) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] should_plug_vifs=False) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'req_state': req_state}) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] time.sleep(sleep_time) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return f(*args, **kwargs) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return self._wait_for_job(job_path) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] error_desc=err_desc) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] During handling of the above exception, another exception occurred: 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Traceback (most recent call last): 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] accel_info=accel_info) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(*args, **kwargs) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] admin_password, network_info, block_device_info) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return function(self, *args, **kwds) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] should_plug_vifs=False) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'req_state': req_state}) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] time.sleep(sleep_time) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self.force_reraise() 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] raise self.value 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return f(*args, **kwargs) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] return self._wait_for_job(job_path) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] error_desc=err_desc) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. 2021-09-10 16:56:42.540 4076 3051112035784 MainThread TRACE nova.compute.manager [instance: 7e8913a3-2544-4f11-9354-908856f92cec] 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.compute.utils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Build of instance 7e8913a3-2544-4f11-9354-908856f92cec was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002e' failed to start. Not enough memory in the system to start the virtual machine instance-0000002e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002e' failed to start. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Not enough memory in the system to start the virtual machine instance-0000002e with ram size 1024 megabytes. (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) 'instance-0000002e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EE466B8-D3BB-4736-9900-FA2EF09DDF1F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:42.569 4076 3051112035784 MainThread DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:43.282 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:44.105 4076 3051112035784 MainThread DEBUG nova.network.neutron [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:44.150 4076 3051112035784 MainThread INFO nova.compute.manager [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] [instance: 7e8913a3-2544-4f11-9354-908856f92cec] Took 1.58 seconds to deallocate network for instance. 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Received event network-vif-plugged-340a8a65-2d59-45c8-9a7d-18033bd9bbe8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Processing event network-vif-plugged-340a8a65-2d59-45c8-9a7d-18033bd9bbe8 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Received event network-vif-plugged-65475916-c478-427b-b1e3-86dd5da029e4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:44.181 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-208abc06-6d27-4195-808f-669f2317c488 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Processing event network-vif-plugged-65475916-c478-427b-b1e3-86dd5da029e4 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:44.213 4076 3051112034376 MainThread DEBUG nova.network.neutron [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Updated VIF entry in instance network info cache for port 7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:44.213 4076 3051112034376 MainThread DEBUG nova.network.neutron [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Updating instance_info_cache with network_info: [{"id": "7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55", "address": "fa:16:3e:8b:69:01", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.155", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7b4ee10d-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:44.245 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Using config drive for instance 2021-09-10 16:56:44.291 4076 3051112035608 MainThread INFO nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Using config drive for instance 2021-09-10 16:56:44.313 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-7c05d77a-e32c-446a-acf7-265c8d23378c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9d815a45-c30c-4ca8-a66d-6ef7fef6185d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:44.520 4076 3051112035784 MainThread INFO nova.scheduler.client.report [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Deleted allocations for instance 7e8913a3-2544-4f11-9354-908856f92cec 2021-09-10 16:56:44.582 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [req-f9008956-59d6-4265-ab4a-989b1b411141 d33d11497a91483d93fe0ed8f5c26603 efc5deaa89294889821f303976e2d776 - default default] Lock "7e8913a3-2544-4f11-9354-908856f92cec" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.687s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:44.645 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Creating config drive at C:\openstack\instances\instance-0000002f\configdrive.iso 2021-09-10 16:56:44.669 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpvbsqvqeo execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:44.832 4076 3051112035608 MainThread INFO nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Creating config drive at C:\openstack\instances\instance-00000030\configdrive.iso 2021-09-10 16:56:44.864 4076 3051112035608 MainThread DEBUG oslo_concurrency.processutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000030\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_ggbobmw execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:45.005 4076 3051053528872 MainThread DEBUG oslo_concurrency.processutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpvbsqvqeo" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:45.190 4076 3051112035608 MainThread DEBUG oslo_concurrency.processutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000030\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_ggbobmw" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:45.433 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Setting boot order for instance: instance-0000002f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:45.479 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:45.510 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '9F6267F9-58E1-4D11-B89D-B03FAA1AF3F3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14868), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:45.622 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Setting boot order for instance: instance-00000030: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:45.678 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:45.697 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': '572CDF3C-1F33-480E-A32F-B4D18DD12C44', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17774), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:45.724 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:45.724 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:45.728 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:45.728 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:45.728 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-bcd1282f-c98a-430e-8db7-0ffaa3dc844b - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:45.728 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-bcd1282f-c98a-430e-8db7-0ffaa3dc844b - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:45.753 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9F6267F9-58E1-4D11-B89D-B03FAA1AF3F3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=152457), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:45.753 4076 3051053528872 MainThread DEBUG os_win._utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.2736387252807617 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:45.780 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-bcd1282f-c98a-430e-8db7-0ffaa3dc844b - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:45.829 4076 3051054571192 MainThread INFO nova.compute.manager [req-bcd1282f-c98a-430e-8db7-0ffaa3dc844b - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:45.854 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '572CDF3C-1F33-480E-A32F-B4D18DD12C44', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=125053), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:45.854 4076 3051112035608 MainThread DEBUG os_win._utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17612600326538086 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:45.854 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:45.854 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:45.854 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:45.854 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:45.879 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-8b56b6f0-27ef-41fe-9519-1d218330d615 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:45.892 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-8b56b6f0-27ef-41fe-9519-1d218330d615 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:45.938 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-8b56b6f0-27ef-41fe-9519-1d218330d615 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:45.970 4076 3051054571192 MainThread INFO nova.compute.manager [req-8b56b6f0-27ef-41fe-9519-1d218330d615 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:46.393 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:46.402 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:46.514 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:46.536 4076 3051112034376 MainThread DEBUG nova.virt.hardware [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:46.536 4076 3051112034376 MainThread INFO nova.compute.claims [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Claim successful on node n-h1-808474-1 2021-09-10 16:56:46.675 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.hostops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:46.700 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.hostops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:46.775 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': 'F9EDE065-F51F-419E-B0D5-13E6C6A56051', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18187), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:46.890 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': '260FA586-8824-44E3-A255-0EBD2197C1DB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16495), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:46.903 4076 3051112034376 MainThread DEBUG nova.compute.provider_tree [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:46.927 4076 3051112034376 MainThread DEBUG nova.scheduler.client.report [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:46.927 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:46.942 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:46.968 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:46.983 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:46.983 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:46.983 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:46.995 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-1db8340b-6667-4a9a-b8cf-76011e53cf26 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:46.995 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-1db8340b-6667-4a9a-b8cf-76011e53cf26 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:47.047 4076 3051112034376 MainThread DEBUG nova.compute.utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:47.047 4076 3051112037896 GreenThread-36 DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:47.047 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:47.047 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9EDE065-F51F-419E-B0D5-13E6C6A56051', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=152093), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:47.047 4076 3051053528872 MainThread DEBUG os_win._utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.5676467418670654 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:47.063 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-1db8340b-6667-4a9a-b8cf-76011e53cf26 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:47.063 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:47.077 4076 3051112035432 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:47.077 4076 3051112035432 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:47.077 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:47.077 4076 3051112035784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:47.135 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '260FA586-8824-44E3-A255-0EBD2197C1DB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=159710), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:47.135 4076 3051112035608 MainThread DEBUG os_win._utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4575510025024414 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:47.160 4076 3051054571192 MainThread INFO nova.compute.manager [req-1db8340b-6667-4a9a-b8cf-76011e53cf26 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:47.160 4076 3051112035432 MainThread DEBUG nova.compute.manager [req-8280f2d7-6b3b-4f45-96de-0b975bdce5e3 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:47.160 4076 3051112035432 MainThread DEBUG nova.virt.hyperv.vmops [req-8280f2d7-6b3b-4f45-96de-0b975bdce5e3 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:47.212 4076 3051112035432 MainThread DEBUG nova.compute.manager [req-8280f2d7-6b3b-4f45-96de-0b975bdce5e3 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:47.239 4076 3051112035432 MainThread INFO nova.compute.manager [req-8280f2d7-6b3b-4f45-96de-0b975bdce5e3 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:47.285 4076 3051112037896 GreenThread-36 DEBUG nova.policy [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '52b9f83ffb604b3388b2c32b65a2e8ff', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e36daecbd54a4138802bd46bc59ade0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:47.301 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:47.317 4076 3051112034376 MainThread INFO nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Spawning new instance 2021-09-10 16:56:47.333 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:47.333 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:47.333 4076 3051112034376 MainThread DEBUG nova.block_device [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:47.333 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:47.333 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:47.349 4076 3051112034376 MainThread DEBUG os_win.utils.pathutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Creating directory: C:\openstack\instances\instance-00000031 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:47.349 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000031\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:47.380 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Resizing VHD C:\openstack\instances\instance-00000031\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:47.792 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-5ea33087-9ab0-4887-a491-eff8417f6a0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Received event network-vif-plugged-7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:47.792 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-5ea33087-9ab0-4887-a491-eff8417f6a0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:47.792 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-5ea33087-9ab0-4887-a491-eff8417f6a0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:47.792 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-5ea33087-9ab0-4887-a491-eff8417f6a0b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Processing event network-vif-plugged-7b4ee10d-c0df-4c56-a1ab-d3c76dca2b55 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:47.849 4076 3051112036488 MainThread INFO nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Using config drive for instance 2021-09-10 16:56:48.072 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '781E2EB8-1911-4DFF-B229-E4C10C178489', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14799), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:48.154 4076 3051112036488 MainThread INFO nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Creating config drive at C:\openstack\instances\instance-0000002d\configdrive.iso 2021-09-10 16:56:48.182 4076 3051112036488 MainThread DEBUG oslo_concurrency.processutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpp6nfcvff execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:48.334 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Successfully created port: 99378947-24eb-49f1-b530-d2511c924e32 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:48.342 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:48.370 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:48.370 4076 3051112038072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:48.370 4076 3051112038072 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:48.370 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:48.370 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:48.393 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '781E2EB8-1911-4DFF-B229-E4C10C178489', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=146929), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:48.393 4076 3051053528872 MainThread DEBUG os_win._utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.9134740829467773 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:48.393 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7D73533E-F5BA-4E80-84DD-21428440A133', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=167958), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:48.393 4076 3051112035608 MainThread DEBUG os_win._utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.715139627456665 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:48.393 4076 3051113087048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:48.393 4076 3051113087048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:48.393 4076 3051112038248 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:48.393 4076 3051112038248 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:48.414 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-d35eccef-a418-4df6-a5db-f70cb7a9dc60 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:48.414 4076 3051112038072 MainThread DEBUG nova.virt.hyperv.vmops [req-d35eccef-a418-4df6-a5db-f70cb7a9dc60 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:48.469 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-d35eccef-a418-4df6-a5db-f70cb7a9dc60 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:48.504 4076 3051112038072 MainThread INFO nova.compute.manager [req-d35eccef-a418-4df6-a5db-f70cb7a9dc60 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:48.504 4076 3051112036488 MainThread DEBUG oslo_concurrency.processutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000002d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpp6nfcvff" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:48.541 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-724e3a86-81b3-4b72-892d-c2b14736293f - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:48.541 4076 3051112038248 MainThread DEBUG nova.virt.hyperv.vmops [req-724e3a86-81b3-4b72-892d-c2b14736293f - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:48.572 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:48.572 4076 3051112036136 MainThread DEBUG nova.virt.hardware [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:48.572 4076 3051112036136 MainThread INFO nova.compute.claims [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Claim successful on node n-h1-808474-1 2021-09-10 16:56:48.592 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-724e3a86-81b3-4b72-892d-c2b14736293f - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:48.631 4076 3051112038248 MainThread INFO nova.compute.manager [req-724e3a86-81b3-4b72-892d-c2b14736293f - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:48.793 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.hostops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:48.793 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.hostops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:48.916 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Setting boot order for instance: instance-0000002d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:48.984 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:49.020 4076 3051112036136 MainThread DEBUG nova.compute.provider_tree [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:49.020 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': '15D369EE-3757-450C-829D-FC6CB929571C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=25459), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:49.034 4076 3051112036136 MainThread DEBUG nova.scheduler.client.report [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:49.034 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.034 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:49.107 4076 3051112036136 MainThread DEBUG nova.compute.utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:49.124 4076 3051112035784 GreenThread-37 DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:49.124 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:49.144 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:49.153 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:49.153 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:49.153 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:49.153 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.170 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '15D369EE-3757-450C-829D-FC6CB929571C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114708), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:49.170 4076 3051112036488 MainThread DEBUG os_win._utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1859912872314453 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:49.207 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-44941428-fd33-416b-b917-a6e1627b2476 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:49.207 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-44941428-fd33-416b-b917-a6e1627b2476 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:49.255 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-44941428-fd33-416b-b917-a6e1627b2476 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:49.277 4076 3051112037720 MainThread INFO nova.compute.manager [req-44941428-fd33-416b-b917-a6e1627b2476 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:49.389 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:49.389 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Successfully updated port: 99378947-24eb-49f1-b530-d2511c924e32 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:49.389 4076 3051112036136 MainThread INFO nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Spawning new instance 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG nova.block_device [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.405 4076 3051112037896 GreenThread-36 DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Acquired lock "refresh_cache-8636793a-b94f-447d-a53a-4506e8a3032f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:49.405 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG os_win.utils.pathutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Creating directory: C:\openstack\instances\instance-00000032 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:49.405 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000032\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:49.449 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': '3B8F0479-CBF7-4540-B7C5-1355D88C526C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22691), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:49.449 4076 3051112035784 GreenThread-37 DEBUG nova.policy [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '4b444bbec44249dda082cd505f5a9c27', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2902d375c2104912b7ea540517edd89f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:49.449 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': 'CFC1E5DE-DF28-4597-B6E6-ED36EE17CA16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21567), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:49.449 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Resizing VHD C:\openstack\instances\instance-00000032\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:49.536 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:49.574 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:49.574 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:49.574 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:49.574 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.588 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3B8F0479-CBF7-4540-B7C5-1355D88C526C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113630), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:49.588 4076 3051112035608 MainThread DEBUG os_win._utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.909898281097412 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:49.588 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-0d2f2f8b-b7db-4a90-b929-1468cdf67f92 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:49.588 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2f2f8b-b7db-4a90-b929-1468cdf67f92 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:49.634 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-0d2f2f8b-b7db-4a90-b929-1468cdf67f92 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:49.656 4076 3051112037720 MainThread INFO nova.compute.manager [req-0d2f2f8b-b7db-4a90-b929-1468cdf67f92 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:49.718 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CFC1E5DE-DF28-4597-B6E6-ED36EE17CA16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=260402), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:49.718 4076 3051053528872 MainThread DEBUG os_win._utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.238619804382324 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:49.732 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:49.732 4076 3051112038248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:49.732 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:49.732 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:49.763 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-db8af1e4-375b-4493-b6f6-4f3212ffdd5f - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:49.763 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-db8af1e4-375b-4493-b6f6-4f3212ffdd5f - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:49.813 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-db8af1e4-375b-4493-b6f6-4f3212ffdd5f - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:49.841 4076 3051112037720 MainThread INFO nova.compute.manager [req-db8af1e4-375b-4493-b6f6-4f3212ffdd5f - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:50.192 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': 'CBDC35B5-7DE0-4D1E-8A96-C0CD8AEADF9E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15795), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:50.203 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Received event network-changed-99378947-24eb-49f1-b530-d2511c924e32 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:50.203 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Refreshing instance network info cache due to event network-changed-99378947-24eb-49f1-b530-d2511c924e32. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:50.317 4076 3051112038248 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:50.317 4076 3051112038248 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:50.317 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:50.317 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:50.332 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CBDC35B5-7DE0-4D1E-8A96-C0CD8AEADF9E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111330), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:50.332 4076 3051112036488 MainThread DEBUG os_win._utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3477635383605957 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:50.342 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-a3526c25-3542-437b-8786-50bd50bba0c2 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:50.342 4076 3051112038248 MainThread DEBUG nova.virt.hyperv.vmops [req-a3526c25-3542-437b-8786-50bd50bba0c2 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:50.387 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-a3526c25-3542-437b-8786-50bd50bba0c2 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:50.387 4076 3051112037896 GreenThread-36 DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Updating instance_info_cache with network_info: [{"id": "99378947-24eb-49f1-b530-d2511c924e32", "address": "fa:16:3e:ca:92:03", "network": {"id": "40dd43aa-44cf-4bf2-851b-fed44406f736", "bridge": null, "label": "tempest-ImagesOneServerNegativeTestJSON-842854431-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e36daecbd54a4138802bd46bc59ade0d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap99378947-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:50.411 4076 3051112038248 MainThread INFO nova.compute.manager [req-a3526c25-3542-437b-8786-50bd50bba0c2 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:50.427 4076 3051112037896 GreenThread-36 DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Releasing lock "refresh_cache-8636793a-b94f-447d-a53a-4506e8a3032f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:50.427 4076 3051112037896 GreenThread-36 DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Instance network_info: |[{"id": "99378947-24eb-49f1-b530-d2511c924e32", "address": "fa:16:3e:ca:92:03", "network": {"id": "40dd43aa-44cf-4bf2-851b-fed44406f736", "bridge": null, "label": "tempest-ImagesOneServerNegativeTestJSON-842854431-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e36daecbd54a4138802bd46bc59ade0d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap99378947-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:50.427 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Preparing to wait for external event network-vif-plugged-99378947-24eb-49f1-b530-d2511c924e32 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:50.427 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:50.427 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:50.427 4076 3051112034376 MainThread DEBUG os_win.utils.compute.vmutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Creating VM instance-00000031 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:50.427 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-8636793a-b94f-447d-a53a-4506e8a3032f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:50.427 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Refreshing network info cache for port 99378947-24eb-49f1-b530-d2511c924e32 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:50.612 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': 'ADC6E498-A592-4F11-BBBB-C165C8BA48DF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16381), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:50.743 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '36D14D48-DAC0-4ECB-BCAD-226CC9240C85', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15488), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:50.763 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'ADC6E498-A592-4F11-BBBB-C165C8BA48DF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116443), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:50.763 4076 3051112035608 MainThread DEBUG os_win._utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.084805250167847 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:50.763 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:50.763 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:50.763 4076 3051112038248 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:50.763 4076 3051112038248 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:50.775 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-d2bd9e3c-f03f-4c9b-a65b-fdc4aedb05ec - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:50.775 4076 3051112038248 MainThread DEBUG nova.virt.hyperv.vmops [req-d2bd9e3c-f03f-4c9b-a65b-fdc4aedb05ec - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:50.824 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-d2bd9e3c-f03f-4c9b-a65b-fdc4aedb05ec - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:50.863 4076 3051112038248 MainThread INFO nova.compute.manager [req-d2bd9e3c-f03f-4c9b-a65b-fdc4aedb05ec - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:50.950 4076 3051112038072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:50.950 4076 3051112038072 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:50.950 4076 3051113132104 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:50.950 4076 3051113132104 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:50.973 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-909ed20c-ac7a-4e75-b4f6-fd053d4e3b40 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:50.973 4076 3051112038072 MainThread DEBUG nova.virt.hyperv.vmops [req-909ed20c-ac7a-4e75-b4f6-fd053d4e3b40 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:51.002 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '36D14D48-DAC0-4ECB-BCAD-226CC9240C85', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=152879), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:51.002 4076 3051053528872 MainThread DEBUG os_win._utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.522573947906494 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:51.016 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-909ed20c-ac7a-4e75-b4f6-fd053d4e3b40 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:51.044 4076 3051112038072 MainThread INFO nova.compute.manager [req-909ed20c-ac7a-4e75-b4f6-fd053d4e3b40 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:51.380 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': '8A8F0A29-DB8F-4EA0-95E5-E34791ACC6B3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15408), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:51.386 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:51.436 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Successfully created port: 850df7b1-8ded-4e6b-b6a7-03c93e659d28 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:51.509 4076 3051112038248 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:51.509 4076 3051112038248 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:51.509 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:51.509 4076 3051112038072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:51.535 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8A8F0A29-DB8F-4EA0-95E5-E34791ACC6B3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114074), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:51.535 4076 3051112036488 MainThread DEBUG os_win._utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.550295114517212 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:51.535 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-b4ffe924-404e-4c51-ba6c-ba3a5cb64b20 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:51.535 4076 3051112038248 MainThread DEBUG nova.virt.hyperv.vmops [req-b4ffe924-404e-4c51-ba6c-ba3a5cb64b20 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:51.595 4076 3051112038248 MainThread DEBUG nova.compute.manager [req-b4ffe924-404e-4c51-ba6c-ba3a5cb64b20 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:51.621 4076 3051112038248 MainThread INFO nova.compute.manager [req-b4ffe924-404e-4c51-ba6c-ba3a5cb64b20 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:51.792 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': '13490BD2-B307-48E0-AF89-F4F04B512FAD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14657), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:51.817 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Updated VIF entry in instance network info cache for port 99378947-24eb-49f1-b530-d2511c924e32. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:51.817 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Updating instance_info_cache with network_info: [{"id": "99378947-24eb-49f1-b530-d2511c924e32", "address": "fa:16:3e:ca:92:03", "network": {"id": "40dd43aa-44cf-4bf2-851b-fed44406f736", "bridge": null, "label": "tempest-ImagesOneServerNegativeTestJSON-842854431-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e36daecbd54a4138802bd46bc59ade0d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap99378947-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:51.833 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-f19aded8-9868-43ad-a376-b7dc3750cd11 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-8636793a-b94f-447d-a53a-4506e8a3032f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:51.937 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:51.937 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:51.937 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:51.937 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: a1386320-87c7-493d-a393-4e590b06faa0] VM Stopped (Lifecycle Event) 2021-09-10 16:56:51.937 4076 3051112035608 MainThread ERROR os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000030\' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A3210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000030\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)instance-000000306B847904-9ACC-434C-8511-8245B2433A32%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '13490BD2-B307-48E0-AF89-F4F04B512FAD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120177), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\nNot enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)\n\n'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32)", 'ErrorSummaryDescription': "'instance-00000030' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000030.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:51.937 4076 3051112035608 MainThread ERROR nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Failed to change vm state of instance-00000030 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. 2021-09-10 16:56:51.937 4076 3051112035608 MainThread INFO nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Got request to destroy instance 2021-09-10 16:56:51.968 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-3d4562cc-d3a3-4b6e-9921-bf82e9752b99 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:51.968 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-3d4562cc-d3a3-4b6e-9921-bf82e9752b99 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:52.017 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-3d4562cc-d3a3-4b6e-9921-bf82e9752b99 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:52.017 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': '39149B12-D93D-4BE8-A7DC-5F7110041434', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14871), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:52.048 4076 3051112037720 MainThread INFO nova.compute.manager [req-3d4562cc-d3a3-4b6e-9921-bf82e9752b99 - - - - -] [instance: a1386320-87c7-493d-a393-4e590b06faa0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:52.212 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:52.212 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:52.212 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:52.212 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] VM Stopped (Lifecycle Event) 2021-09-10 16:56:52.243 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-9c8c018c-e941-4755-8ba8-66c6f3670b27 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:52.243 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-9c8c018c-e941-4755-8ba8-66c6f3670b27 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:52.275 4076 3051053528872 MainThread ERROR os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002f\' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C86510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)instance-0000002f8B5235F7-7140-4411-94FF-47C8F5D9C865%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '39149B12-D93D-4BE8-A7DC-5F7110041434', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=148389), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\nNot enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)\n\n'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865)", 'ErrorSummaryDescription': "'instance-0000002f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:52.275 4076 3051053528872 MainThread ERROR nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Failed to change vm state of instance-0000002f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. 2021-09-10 16:56:52.275 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Got request to destroy instance 2021-09-10 16:56:52.293 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-9c8c018c-e941-4755-8ba8-66c6f3670b27 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:52.293 4076 3051112037720 MainThread INFO nova.compute.manager [req-9c8c018c-e941-4755-8ba8-66c6f3670b27 - - - - -] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:52.556 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': '67C70663-419A-4AA0-A71A-3E9E2833FED4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15513), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:52.705 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:52.705 4076 3051112037896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:52.705 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:52.705 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:52.714 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '67C70663-419A-4AA0-A71A-3E9E2833FED4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115216), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:52.714 4076 3051112036488 MainThread DEBUG os_win._utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.729145050048828 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:52.714 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Successfully updated port: 850df7b1-8ded-4e6b-b6a7-03c93e659d28 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:52.746 4076 3051112035784 GreenThread-37 DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Acquired lock "refresh_cache-cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:52.746 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:52.763 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-9cc1e1d7-6d6f-4874-a2b2-4ca21185711a - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:52.786 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-9cc1e1d7-6d6f-4874-a2b2-4ca21185711a - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:52.833 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-9cc1e1d7-6d6f-4874-a2b2-4ca21185711a - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:52.849 4076 3051112037720 MainThread INFO nova.compute.manager [req-9cc1e1d7-6d6f-4874-a2b2-4ca21185711a - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:52.893 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Received event network-changed-850df7b1-8ded-4e6b-b6a7-03c93e659d28 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:52.893 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Refreshing instance network info cache due to event network-changed-850df7b1-8ded-4e6b-b6a7-03c93e659d28. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:56:53.327 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:53.745 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': '848AE3A6-4B79-4AEA-815E-28BF186DFB23', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19469), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:53.886 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:53.886 4076 3051112035432 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:53.886 4076 3051112038072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:53.886 4076 3051112038072 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:53.911 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '848AE3A6-4B79-4AEA-815E-28BF186DFB23', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=125009), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:53.911 4076 3051112036488 MainThread DEBUG os_win._utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.926411390304565 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:53.911 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:53.911 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:53.911 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:53.919 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-66c01bd7-3faf-4ab0-bc55-103234f2c091 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:53.919 4076 3051112038072 MainThread DEBUG nova.virt.hyperv.vmops [req-66c01bd7-3faf-4ab0-bc55-103234f2c091 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:53.933 4076 3051112035608 MainThread DEBUG os_win.utils.compute.vmutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Successfully changed vm state of instance-00000030 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:53.933 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Successfully changed state of VM instance-00000030 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:53.980 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-66c01bd7-3faf-4ab0-bc55-103234f2c091 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:53.980 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Waiting for WMI job: {'InstanceID': '5069B797-0C8E-4BF0-A3F2-A4FD6BAA8A34', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18158), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:54.005 4076 3051112038072 MainThread INFO nova.compute.manager [req-66c01bd7-3faf-4ab0-bc55-103234f2c091 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:54.056 4076 3051112035784 GreenThread-37 DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Updating instance_info_cache with network_info: [{"id": "850df7b1-8ded-4e6b-b6a7-03c93e659d28", "address": "fa:16:3e:c8:4b:ea", "network": {"id": "c41f1f8c-6dd1-48a5-8a0a-87f80e196ac9", "bridge": null, "label": "tempest-ServersTestFqdnHostnames-712389274-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "2902d375c2104912b7ea540517edd89f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap850df7b1-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:54.088 4076 3051112035784 GreenThread-37 DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Releasing lock "refresh_cache-cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:54.088 4076 3051112035784 GreenThread-37 DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Instance network_info: |[{"id": "850df7b1-8ded-4e6b-b6a7-03c93e659d28", "address": "fa:16:3e:c8:4b:ea", "network": {"id": "c41f1f8c-6dd1-48a5-8a0a-87f80e196ac9", "bridge": null, "label": "tempest-ServersTestFqdnHostnames-712389274-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "2902d375c2104912b7ea540517edd89f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap850df7b1-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:54.088 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Preparing to wait for external event network-vif-plugged-850df7b1-8ded-4e6b-b6a7-03c93e659d28 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:54.088 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.088 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.088 4076 3051112036136 MainThread DEBUG os_win.utils.compute.vmutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Creating VM instance-00000032 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:56:54.088 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:54.088 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Refreshing network info cache for port 850df7b1-8ded-4e6b-b6a7-03c93e659d28 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:56:54.088 4076 3051112035608 MainThread DEBUG os_win.utils.jobutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] WMI job succeeded: {'InstanceID': '5069B797-0C8E-4BF0-A3F2-A4FD6BAA8A34', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32239), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:54.088 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.088 4076 3051112035608 MainThread DEBUG os_win.utils.pathutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Removing directory: C:\openstack\instances\instance-00000030 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:54.108 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.108 4076 3051112035608 MainThread ERROR nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Traceback (most recent call last): 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(*args, **kwargs) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(self, *args, **kwds) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] should_plug_vifs=False) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'req_state': req_state}) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] time.sleep(sleep_time) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return f(*args, **kwargs) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return self._wait_for_job(job_path) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] error_desc=err_desc) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] During handling of the above exception, another exception occurred: 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Traceback (most recent call last): 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] yield resources 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] accel_info=accel_info) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(*args, **kwargs) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(self, *args, **kwds) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] should_plug_vifs=False) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'req_state': req_state}) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] time.sleep(sleep_time) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return f(*args, **kwargs) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return self._wait_for_job(job_path) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] error_desc=err_desc) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.108 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.108 4076 3051112035608 MainThread INFO nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Terminating instance 2021-09-10 16:56:54.108 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:54.108 4076 3051112035608 MainThread INFO nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Got request to destroy instance 2021-09-10 16:56:54.126 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.vmops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:54.126 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.126 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "instance-00000030" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.126 4076 3051112035608 MainThread INFO nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:54.140 4076 3051112035608 MainThread DEBUG nova.compute.claims [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:54.140 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.246 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.hostops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:54.263 4076 3051112035608 MainThread DEBUG nova.virt.hyperv.hostops [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:54.432 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:54.432 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.432 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.432 4076 3051053528872 MainThread DEBUG os_win.utils.compute.vmutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Successfully changed vm state of instance-0000002f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:54.432 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Successfully changed state of VM instance-0000002f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:54.496 4076 3051112035608 MainThread DEBUG nova.compute.provider_tree [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:54.519 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Waiting for WMI job: {'InstanceID': 'B497EC11-6211-49DF-BFDF-6DA80EB1B15B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19996), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.528 4076 3051112035608 MainThread ERROR nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Traceback (most recent call last): 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(*args, **kwargs) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(self, *args, **kwds) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] should_plug_vifs=False) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'req_state': req_state}) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] time.sleep(sleep_time) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return f(*args, **kwargs) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return self._wait_for_job(job_path) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] error_desc=err_desc) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] During handling of the above exception, another exception occurred: 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Traceback (most recent call last): 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] accel_info=accel_info) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(*args, **kwargs) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return function(self, *args, **kwds) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] should_plug_vifs=False) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'req_state': req_state}) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] time.sleep(sleep_time) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self.force_reraise() 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] raise self.value 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return f(*args, **kwargs) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] return self._wait_for_job(job_path) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] error_desc=err_desc) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. 2021-09-10 16:56:54.528 4076 3051112035608 MainThread TRACE nova.compute.manager [instance: a1386320-87c7-493d-a393-4e590b06faa0] 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG nova.compute.utils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Build of instance a1386320-87c7-493d-a393-4e590b06faa0 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000030' failed to start. Not enough memory in the system to start the virtual machine instance-00000030. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000030' failed to start. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Not enough memory in the system to start the virtual machine instance-00000030 with ram size 1024 megabytes. (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) 'instance-00000030' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B847904-9ACC-434C-8511-8245B2433A32) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:54.528 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:54.542 4076 3051112035608 MainThread DEBUG nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:54.542 4076 3051112035608 MainThread DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:54.645 4076 3051053528872 MainThread DEBUG os_win.utils.jobutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] WMI job succeeded: {'InstanceID': 'B497EC11-6211-49DF-BFDF-6DA80EB1B15B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=39992), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:54.645 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.645 4076 3051053528872 MainThread DEBUG os_win.utils.pathutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Removing directory: C:\openstack\instances\instance-0000002f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:54.645 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.645 4076 3051053528872 MainThread ERROR nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Traceback (most recent call last): 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(*args, **kwargs) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(self, *args, **kwds) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] should_plug_vifs=False) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'req_state': req_state}) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] time.sleep(sleep_time) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return f(*args, **kwargs) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return self._wait_for_job(job_path) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] error_desc=err_desc) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During handling of the above exception, another exception occurred: 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Traceback (most recent call last): 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] yield resources 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] accel_info=accel_info) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(*args, **kwargs) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] admin_password, network_info, block_device_info) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(self, *args, **kwds) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] should_plug_vifs=False) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'req_state': req_state}) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] time.sleep(sleep_time) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return f(*args, **kwargs) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return self._wait_for_job(job_path) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] error_desc=err_desc) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:54.645 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:54.645 4076 3051053528872 MainThread INFO nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Terminating instance 2021-09-10 16:56:54.645 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:54.645 4076 3051053528872 MainThread INFO nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Got request to destroy instance 2021-09-10 16:56:54.661 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:54.661 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.661 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "instance-0000002f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:54.661 4076 3051053528872 MainThread INFO nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:54.661 4076 3051053528872 MainThread DEBUG nova.compute.claims [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:54.661 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:54.806 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:54.837 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.hostops [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:54.942 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': '0B7C2211-EEF4-440C-A29B-DEE397E85D00', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15992), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:55.056 4076 3051053528872 MainThread DEBUG nova.compute.provider_tree [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:55.056 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.scheduler.client.report [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:55.077 4076 3051053528872 MainThread ERROR nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Traceback (most recent call last): 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(*args, **kwargs) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] admin_password, network_info, block_device_info) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(self, *args, **kwds) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] should_plug_vifs=False) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'req_state': req_state}) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] time.sleep(sleep_time) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return f(*args, **kwargs) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return self._wait_for_job(job_path) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] error_desc=err_desc) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] During handling of the above exception, another exception occurred: 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Traceback (most recent call last): 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] accel_info=accel_info) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(*args, **kwargs) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] admin_password, network_info, block_device_info) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return function(self, *args, **kwds) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] should_plug_vifs=False) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'req_state': req_state}) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] time.sleep(sleep_time) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self.force_reraise() 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] raise self.value 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return f(*args, **kwargs) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] return self._wait_for_job(job_path) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] error_desc=err_desc) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. 2021-09-10 16:56:55.077 4076 3051053528872 MainThread TRACE nova.compute.manager [instance: a5380419-a846-48fc-891c-d88dc8215cfd] 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.compute.utils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Build of instance a5380419-a846-48fc-891c-d88dc8215cfd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002f' failed to start. Not enough memory in the system to start the virtual machine instance-0000002f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002f' failed to start. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Not enough memory in the system to start the virtual machine instance-0000002f with ram size 1024 megabytes. (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) 'instance-0000002f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8B5235F7-7140-4411-94FF-47C8F5D9C865) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:55.077 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:55.120 4076 3051112036488 MainThread ERROR os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job failed: {'RawErrors': (('124\'instance-0000002d\' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000002d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)instance-0000002d668BFBDF-587A-4B06-A144-02E676831D6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0B7C2211-EEF4-440C-A29B-DEE397E85D00', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115433), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\nNot enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)\n\n'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E)", 'ErrorSummaryDescription': "'instance-0000002d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000002d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:55.120 4076 3051112036488 MainThread ERROR nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Failed to change vm state of instance-0000002d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. 2021-09-10 16:56:55.120 4076 3051112036488 MainThread INFO nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Got request to destroy instance 2021-09-10 16:56:55.120 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:55.120 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] VM Stopped (Lifecycle Event) 2021-09-10 16:56:55.120 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:55.120 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:55.152 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-04b76987-afea-4246-84a5-c08cb8e2d449 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:55.152 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-04b76987-afea-4246-84a5-c08cb8e2d449 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:55.192 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-04b76987-afea-4246-84a5-c08cb8e2d449 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:55.241 4076 3051113162360 MainThread INFO nova.compute.manager [req-04b76987-afea-4246-84a5-c08cb8e2d449 - - - - -] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:55.373 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Updated VIF entry in instance network info cache for port 850df7b1-8ded-4e6b-b6a7-03c93e659d28. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:56:55.373 4076 3051112037720 MainThread DEBUG nova.network.neutron [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Updating instance_info_cache with network_info: [{"id": "850df7b1-8ded-4e6b-b6a7-03c93e659d28", "address": "fa:16:3e:c8:4b:ea", "network": {"id": "c41f1f8c-6dd1-48a5-8a0a-87f80e196ac9", "bridge": null, "label": "tempest-ServersTestFqdnHostnames-712389274-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "2902d375c2104912b7ea540517edd89f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap850df7b1-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:55.393 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-29ea3fed-68a5-4c94-9e3d-082bd568e198 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:55.438 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-56376552-8c76-460d-9114-08cdf4d10489 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Received event network-vif-plugged-99378947-24eb-49f1-b530-d2511c924e32 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:56:55.438 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-56376552-8c76-460d-9114-08cdf4d10489 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:55.438 4076 3051112037720 MainThread DEBUG oslo_concurrency.lockutils [req-56376552-8c76-460d-9114-08cdf4d10489 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:55.438 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-56376552-8c76-460d-9114-08cdf4d10489 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Processing event network-vif-plugged-99378947-24eb-49f1-b530-d2511c924e32 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:56:55.483 4076 3051112034376 MainThread INFO nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Using config drive for instance 2021-09-10 16:56:55.761 4076 3051112034376 MainThread INFO nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Creating config drive at C:\openstack\instances\instance-00000031\configdrive.iso 2021-09-10 16:56:55.799 4076 3051112034376 MainThread DEBUG oslo_concurrency.processutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000031\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa4cybe2c execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:56:55.950 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:55.950 4076 3051112035608 MainThread DEBUG nova.network.neutron [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:55.982 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:56:55.982 4076 3051112035608 MainThread INFO nova.compute.manager [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] [instance: a1386320-87c7-493d-a393-4e590b06faa0] Took 1.44 seconds to deallocate network for instance. 2021-09-10 16:56:56.119 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:56.143 4076 3051113162360 MainThread DEBUG nova.virt.hardware [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:56:56.143 4076 3051113162360 MainThread INFO nova.compute.claims [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Claim successful on node n-h1-808474-1 2021-09-10 16:56:56.154 4076 3051112034376 MainThread DEBUG oslo_concurrency.processutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000031\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa4cybe2c" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:56:56.270 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:56.277 4076 3051112035608 MainThread INFO nova.scheduler.client.report [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Deleted allocations for instance a1386320-87c7-493d-a393-4e590b06faa0 2021-09-10 16:56:56.290 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:56.317 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [req-fe296b6a-f508-4eba-b0fd-894ddac2fc5b a6076248095044dcbd42424ad33c0395 2113aff5a9ed47f5bdb3bdee01c77f67 - default default] Lock "a1386320-87c7-493d-a393-4e590b06faa0" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:56.556 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:56.583 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:56.590 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:56.590 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:56:56.660 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:56:56.674 4076 3051112037720 GreenThread-38 DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:56:56.674 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:56:56.674 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Setting boot order for instance: instance-00000031: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:56:56.713 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:56:56.738 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:56:56.784 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'E023EC6F-72B1-4B78-99A8-143116C8DDD2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20208), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:56.844 4076 3051112037720 GreenThread-38 DEBUG nova.policy [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '42267190de3648eea52604e6b2444d20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6ca07d9dc99a4581a698a232ffa065db', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:56:56.927 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:56.927 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:56.927 4076 3051112038072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:56.927 4076 3051112038072 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:56:56.934 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:56:56.934 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E023EC6F-72B1-4B78-99A8-143116C8DDD2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119261), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:56.934 4076 3051112034376 MainThread DEBUG os_win._utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1952381134033203 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:56.934 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Spawning new instance 2021-09-10 16:56:56.952 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-80e98f81-ade8-48ea-9120-16f60631c5e5 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:56.952 4076 3051112038072 MainThread DEBUG nova.virt.hyperv.vmops [req-80e98f81-ade8-48ea-9120-16f60631c5e5 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:56.952 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:56.952 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:56.952 4076 3051113162360 MainThread DEBUG nova.block_device [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:56:56.952 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:56.952 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:56.973 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating directory: C:\openstack\instances\instance-00000033 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:56:56.973 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000033\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:56:56.999 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Resizing VHD C:\openstack\instances\instance-00000033\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:56:56.999 4076 3051112038072 MainThread DEBUG nova.compute.manager [req-80e98f81-ade8-48ea-9120-16f60631c5e5 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:57.044 4076 3051112038072 MainThread INFO nova.compute.manager [req-80e98f81-ade8-48ea-9120-16f60631c5e5 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:57.410 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:56:57.410 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:57.410 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:57.426 4076 3051112036488 MainThread DEBUG os_win.utils.compute.vmutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Successfully changed vm state of instance-0000002d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:56:57.426 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Successfully changed state of VM instance-0000002d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:56:57.473 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Waiting for WMI job: {'InstanceID': 'D7FEA89E-9008-4E05-A31E-2F7DA6FE5265', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14479), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:57.593 4076 3051112036488 MainThread DEBUG os_win.utils.jobutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] WMI job succeeded: {'InstanceID': 'D7FEA89E-9008-4E05-A31E-2F7DA6FE5265', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28766), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:56:57.593 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:57.593 4076 3051112036488 MainThread DEBUG os_win.utils.pathutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Removing directory: C:\openstack\instances\instance-0000002d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:56:57.593 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:57.593 4076 3051112036488 MainThread ERROR nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Traceback (most recent call last): 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(*args, **kwargs) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] admin_password, network_info, block_device_info) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(self, *args, **kwds) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] should_plug_vifs=False) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'req_state': req_state}) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] time.sleep(sleep_time) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return f(*args, **kwargs) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return self._wait_for_job(job_path) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] error_desc=err_desc) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During handling of the above exception, another exception occurred: 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Traceback (most recent call last): 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] yield resources 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] accel_info=accel_info) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(*args, **kwargs) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] admin_password, network_info, block_device_info) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(self, *args, **kwds) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] should_plug_vifs=False) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'req_state': req_state}) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] time.sleep(sleep_time) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return f(*args, **kwargs) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return self._wait_for_job(job_path) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] error_desc=err_desc) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.593 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.593 4076 3051112036488 MainThread INFO nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Terminating instance 2021-09-10 16:56:57.593 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:56:57.593 4076 3051112036488 MainThread INFO nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Got request to destroy instance 2021-09-10 16:56:57.613 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:56:57.626 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:57.626 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "instance-0000002d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:57.626 4076 3051112036488 MainThread INFO nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:56:57.626 4076 3051112036488 MainThread DEBUG nova.compute.claims [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:56:57.626 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:57.738 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.hostops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:56:57.754 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.hostops [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:56:57.952 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Successfully created port: 644e3ea2-6bb8-448d-91e3-121af5fcdfa2 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:56:57.952 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'B9B38E64-CB70-4F4F-934A-D1901AB643FF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14820), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:57.972 4076 3051112036488 MainThread DEBUG nova.compute.provider_tree [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG nova.scheduler.client.report [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:57.980 4076 3051112036488 MainThread ERROR nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Traceback (most recent call last): 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(*args, **kwargs) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] admin_password, network_info, block_device_info) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(self, *args, **kwds) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] should_plug_vifs=False) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'req_state': req_state}) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] time.sleep(sleep_time) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return f(*args, **kwargs) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return self._wait_for_job(job_path) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] error_desc=err_desc) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] During handling of the above exception, another exception occurred: 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Traceback (most recent call last): 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] accel_info=accel_info) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(*args, **kwargs) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] admin_password, network_info, block_device_info) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return function(self, *args, **kwds) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] should_plug_vifs=False) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'req_state': req_state}) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] time.sleep(sleep_time) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self.force_reraise() 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] raise self.value 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return f(*args, **kwargs) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] return self._wait_for_job(job_path) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] error_desc=err_desc) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. 2021-09-10 16:56:57.980 4076 3051112036488 MainThread TRACE nova.compute.manager [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG nova.compute.utils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Build of instance 9d815a45-c30c-4ca8-a66d-6ef7fef6185d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000002d' failed to start. Not enough memory in the system to start the virtual machine instance-0000002d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000002d' failed to start. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Not enough memory in the system to start the virtual machine instance-0000002d with ram size 1024 megabytes. (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) 'instance-0000002d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 668BFBDF-587A-4B06-A144-02E676831D6E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:56:57.980 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:56:58.002 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:56:58.002 4076 3051112036488 MainThread DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:56:58.072 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:58.072 4076 3051053530104 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:58.072 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:58.072 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:56:58.072 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B9B38E64-CB70-4F4F-934A-D1901AB643FF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111928), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:58.072 4076 3051112034376 MainThread DEBUG os_win._utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3339838981628418 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:58.127 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-e571da97-f1aa-4d63-a1e9-75fbcdf7cae3 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:58.127 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-e571da97-f1aa-4d63-a1e9-75fbcdf7cae3 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:58.176 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-e571da97-f1aa-4d63-a1e9-75fbcdf7cae3 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:58.204 4076 3051092583000 MainThread INFO nova.compute.manager [req-e571da97-f1aa-4d63-a1e9-75fbcdf7cae3 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:58.605 4076 3051053528872 MainThread DEBUG nova.network.neutron [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:58.627 4076 3051053528872 MainThread INFO nova.compute.manager [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] [instance: a5380419-a846-48fc-891c-d88dc8215cfd] Took 3.56 seconds to deallocate network for instance. 2021-09-10 16:56:58.973 4076 3051112036488 MainThread DEBUG nova.network.neutron [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:58.993 4076 3051053528872 MainThread INFO nova.scheduler.client.report [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Deleted allocations for instance a5380419-a846-48fc-891c-d88dc8215cfd 2021-09-10 16:56:59.012 4076 3051112036488 MainThread INFO nova.compute.manager [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] [instance: 9d815a45-c30c-4ca8-a66d-6ef7fef6185d] Took 1.01 seconds to deallocate network for instance. 2021-09-10 16:56:59.083 4076 3051053528872 MainThread DEBUG oslo_concurrency.lockutils [req-43584bb6-e95c-41dd-9f96-67158c553862 73ac9100b70e4fa1b6ad7751b223e5ef 778686f367c642f39349bb208103a3ca - default default] Lock "a5380419-a846-48fc-891c-d88dc8215cfd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 36.890s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:59.089 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': '08B94722-121C-491F-BCD9-7B4E9139A6ED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15426), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:56:59.229 4076 3051054567848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:56:59.229 4076 3051054567848 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:56:59.229 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:59.229 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:59.229 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '08B94722-121C-491F-BCD9-7B4E9139A6ED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110973), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 56, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 56, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:56:59.229 4076 3051112034376 MainThread DEBUG os_win._utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.490755319595337 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:56:59.229 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Successfully updated port: 644e3ea2-6bb8-448d-91e3-121af5fcdfa2 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:56:59.274 4076 3051112037720 GreenThread-38 DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Acquired lock "refresh_cache-19bd864f-b01d-42d3-8ca6-9b75067146fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:56:59.274 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:56:59.280 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-d8fdce32-c0ee-41d3-b670-b71923b53c8f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:56:59.280 4076 3051054567848 MainThread DEBUG nova.virt.hyperv.vmops [req-d8fdce32-c0ee-41d3-b670-b71923b53c8f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:56:59.280 4076 3051112036488 MainThread INFO nova.scheduler.client.report [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Deleted allocations for instance 9d815a45-c30c-4ca8-a66d-6ef7fef6185d 2021-09-10 16:56:59.306 4076 3051054567848 MainThread DEBUG nova.compute.manager [req-d8fdce32-c0ee-41d3-b670-b71923b53c8f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:56:59.348 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-f9922ce4-cb33-4985-b7ba-5ddcbabe7a3d 9bc7e248f98441caa76226a9e8d21746 e122cd09445d480a826d8bd883b15cad - default default] Lock "9d815a45-c30c-4ca8-a66d-6ef7fef6185d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 21.906s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:59.400 4076 3051054567848 MainThread INFO nova.compute.manager [req-d8fdce32-c0ee-41d3-b670-b71923b53c8f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:56:59.400 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:56:59.868 4076 3051112037720 GreenThread-38 DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Updating instance_info_cache with network_info: [{"id": "644e3ea2-6bb8-448d-91e3-121af5fcdfa2", "address": "fa:16:3e:eb:1d:eb", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap644e3ea2-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:56:59.879 4076 3051112037720 GreenThread-38 DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Releasing lock "refresh_cache-19bd864f-b01d-42d3-8ca6-9b75067146fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:56:59.879 4076 3051112037720 GreenThread-38 DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Instance network_info: |[{"id": "644e3ea2-6bb8-448d-91e3-121af5fcdfa2", "address": "fa:16:3e:eb:1d:eb", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap644e3ea2-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:56:59.879 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Preparing to wait for external event network-vif-plugged-644e3ea2-6bb8-448d-91e3-121af5fcdfa2 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:56:59.879 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:56:59.899 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:56:59.899 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating VM instance-00000033 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:57:00.263 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'E6896763-4570-4B46-BC6F-7F6F52DBFA07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15509), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:00.393 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:00.393 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:00.393 4076 3051054571192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:00.393 4076 3051054571192 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:57:00.410 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E6896763-4570-4B46-BC6F-7F6F52DBFA07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117612), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:00.410 4076 3051112034376 MainThread DEBUG os_win._utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6718432903289795 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:00.448 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-8b5a5b44-1d49-450f-8bfa-816223a52b89 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:00.448 4076 3051054571192 MainThread DEBUG nova.virt.hyperv.vmops [req-8b5a5b44-1d49-450f-8bfa-816223a52b89 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:00.513 4076 3051054571192 MainThread DEBUG nova.compute.manager [req-8b5a5b44-1d49-450f-8bfa-816223a52b89 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:00.535 4076 3051054571192 MainThread INFO nova.compute.manager [req-8b5a5b44-1d49-450f-8bfa-816223a52b89 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:00.806 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:57:01.435 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'DFCB9B8E-B4FD-44D5-B326-3224D4131897', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14600), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:01.572 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:01.572 4076 3051054571192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:01.572 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:01.572 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:57:01.572 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DFCB9B8E-B4FD-44D5-B326-3224D4131897', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109378), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:01.572 4076 3051112034376 MainThread DEBUG os_win._utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.833729982376099 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:01.602 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-91cb3671-649b-44fc-8b4a-33d7df0dbe62 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:01.602 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-91cb3671-649b-44fc-8b4a-33d7df0dbe62 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:01.651 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-91cb3671-649b-44fc-8b4a-33d7df0dbe62 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:01.692 4076 3051092583000 MainThread INFO nova.compute.manager [req-91cb3671-649b-44fc-8b4a-33d7df0dbe62 - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:02.041 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-fd998868-355e-4823-a664-085660da7ad9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Received event network-vif-plugged-850df7b1-8ded-4e6b-b6a7-03c93e659d28 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:02.041 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-fd998868-355e-4823-a664-085660da7ad9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:02.041 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-fd998868-355e-4823-a664-085660da7ad9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:02.041 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-fd998868-355e-4823-a664-085660da7ad9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Processing event network-vif-plugged-850df7b1-8ded-4e6b-b6a7-03c93e659d28 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:57:02.082 4076 3051112036136 MainThread INFO nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Using config drive for instance 2021-09-10 16:57:02.090 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Received event network-changed-644e3ea2-6bb8-448d-91e3-121af5fcdfa2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:02.090 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Refreshing instance network info cache due to event network-changed-644e3ea2-6bb8-448d-91e3-121af5fcdfa2. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:57:02.090 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-19bd864f-b01d-42d3-8ca6-9b75067146fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:02.090 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Refreshing network info cache for port 644e3ea2-6bb8-448d-91e3-121af5fcdfa2 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:57:02.493 4076 3051112036136 MainThread INFO nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Creating config drive at C:\openstack\instances\instance-00000032\configdrive.iso 2021-09-10 16:57:02.536 4076 3051112036136 MainThread DEBUG oslo_concurrency.processutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000032\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpzjldmolp execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:57:02.701 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'EA72867F-79A4-4E75-A5FE-CD46C1A6677F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14454), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:02.829 4076 3051112037720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:02.829 4076 3051112037720 MainThread INFO nova.compute.manager [-] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] VM Stopped (Lifecycle Event) 2021-09-10 16:57:02.849 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:02.849 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:02.849 4076 3051112034376 MainThread ERROR os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000031\' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000031\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)instance-00000031292D1599-0940-4B99-A4FA-04DFB11EFCA5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EA72867F-79A4-4E75-A5FE-CD46C1A6677F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111574), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\nNot enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)\n\n'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5)", 'ErrorSummaryDescription': "'instance-00000031' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000031.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:02.849 4076 3051112034376 MainThread ERROR nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Failed to change vm state of instance-00000031 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. 2021-09-10 16:57:02.849 4076 3051112034376 MainThread INFO nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Got request to destroy instance 2021-09-10 16:57:02.860 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-e779abfe-2342-4fcb-a046-887d8e415c5f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:02.860 4076 3051112037720 MainThread DEBUG nova.virt.hyperv.vmops [req-e779abfe-2342-4fcb-a046-887d8e415c5f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:02.860 4076 3051112036136 MainThread DEBUG oslo_concurrency.processutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000032\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpzjldmolp" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:57:02.922 4076 3051112037720 MainThread DEBUG nova.compute.manager [req-e779abfe-2342-4fcb-a046-887d8e415c5f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:02.965 4076 3051112037720 MainThread INFO nova.compute.manager [req-e779abfe-2342-4fcb-a046-887d8e415c5f - - - - -] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:03.276 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Setting boot order for instance: instance-00000032: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:57:03.339 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:57:03.339 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Updated VIF entry in instance network info cache for port 644e3ea2-6bb8-448d-91e3-121af5fcdfa2. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:57:03.339 4076 3051092583000 MainThread DEBUG nova.network.neutron [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Updating instance_info_cache with network_info: [{"id": "644e3ea2-6bb8-448d-91e3-121af5fcdfa2", "address": "fa:16:3e:eb:1d:eb", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap644e3ea2-6b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:03.371 4076 3051092583000 MainThread DEBUG oslo_concurrency.lockutils [req-31b6ed85-250f-4465-8fe8-144df4e1c306 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-19bd864f-b01d-42d3-8ca6-9b75067146fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:03.371 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': 'EF89C191-AEBA-46FD-B589-BA9C5E1DCB99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16805), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:03.515 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:03.515 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:03.515 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:03.515 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:03.532 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EF89C191-AEBA-46FD-B589-BA9C5E1DCB99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116388), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:03.532 4076 3051112036136 MainThread DEBUG os_win._utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19270682334899902 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:03.546 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a7089eaf-d86d-470a-9f2f-7302309cd7ca - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:03.546 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-a7089eaf-d86d-470a-9f2f-7302309cd7ca - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:03.590 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-a7089eaf-d86d-470a-9f2f-7302309cd7ca - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:03.622 4076 3051092583000 MainThread INFO nova.compute.manager [req-a7089eaf-d86d-470a-9f2f-7302309cd7ca - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:04.566 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': 'A2E1467C-2BBA-4F63-A9FD-CBF3713A9F62', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16223), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:04.712 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:04.712 4076 3051054567848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:04.712 4076 3051092583000 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:04.712 4076 3051092583000 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:04.712 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A2E1467C-2BBA-4F63-A9FD-CBF3713A9F62', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122117), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:04.712 4076 3051112036136 MainThread DEBUG os_win._utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3729288578033447 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:04.739 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-d2b83753-4bee-4581-8861-5d76c0f14ac1 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:04.739 4076 3051092583000 MainThread DEBUG nova.virt.hyperv.vmops [req-d2b83753-4bee-4581-8861-5d76c0f14ac1 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:04.756 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:57:04.756 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:04.756 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:04.765 4076 3051112034376 MainThread DEBUG os_win.utils.compute.vmutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Successfully changed vm state of instance-00000031 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:57:04.765 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Successfully changed state of VM instance-00000031 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:57:04.784 4076 3051092583000 MainThread DEBUG nova.compute.manager [req-d2b83753-4bee-4581-8861-5d76c0f14ac1 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:04.807 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Waiting for WMI job: {'InstanceID': 'DB9E8DBE-D02B-438E-B237-82B8635A5629', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14905), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:04.843 4076 3051092583000 MainThread INFO nova.compute.manager [req-d2b83753-4bee-4581-8861-5d76c0f14ac1 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread DEBUG os_win.utils.jobutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] WMI job succeeded: {'InstanceID': 'DB9E8DBE-D02B-438E-B237-82B8635A5629', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29282), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:57:04.922 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:04.922 4076 3051112034376 MainThread DEBUG os_win.utils.pathutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Removing directory: C:\openstack\instances\instance-00000031 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:57:04.922 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:04.922 4076 3051112034376 MainThread ERROR nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Traceback (most recent call last): 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(*args, **kwargs) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] admin_password, network_info, block_device_info) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(self, *args, **kwds) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] should_plug_vifs=False) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'req_state': req_state}) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] time.sleep(sleep_time) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return f(*args, **kwargs) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return self._wait_for_job(job_path) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] error_desc=err_desc) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During handling of the above exception, another exception occurred: 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Traceback (most recent call last): 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] yield resources 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] accel_info=accel_info) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(*args, **kwargs) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] admin_password, network_info, block_device_info) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(self, *args, **kwds) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] should_plug_vifs=False) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'req_state': req_state}) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] time.sleep(sleep_time) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return f(*args, **kwargs) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return self._wait_for_job(job_path) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] error_desc=err_desc) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:04.922 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:04.922 4076 3051112034376 MainThread INFO nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Terminating instance 2021-09-10 16:57:04.922 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:57:04.922 4076 3051112034376 MainThread INFO nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Got request to destroy instance 2021-09-10 16:57:04.941 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.vmops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:57:04.941 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:04.941 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "instance-00000031" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:04.941 4076 3051112034376 MainThread INFO nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:57:04.941 4076 3051112034376 MainThread DEBUG nova.compute.claims [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:57:04.941 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:05.161 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.hostops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:05.176 4076 3051112034376 MainThread DEBUG nova.virt.hyperv.hostops [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:05.384 4076 3051112034376 MainThread DEBUG nova.compute.provider_tree [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:05.384 4076 3051112034376 MainThread DEBUG nova.scheduler.client.report [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:05.384 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:05.411 4076 3051112034376 MainThread ERROR nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Traceback (most recent call last): 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(*args, **kwargs) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] admin_password, network_info, block_device_info) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(self, *args, **kwds) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] should_plug_vifs=False) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'req_state': req_state}) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] time.sleep(sleep_time) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return f(*args, **kwargs) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return self._wait_for_job(job_path) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] error_desc=err_desc) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] During handling of the above exception, another exception occurred: 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Traceback (most recent call last): 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] accel_info=accel_info) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(*args, **kwargs) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] admin_password, network_info, block_device_info) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return function(self, *args, **kwds) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] should_plug_vifs=False) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'req_state': req_state}) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] time.sleep(sleep_time) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self.force_reraise() 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] raise self.value 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return f(*args, **kwargs) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] return self._wait_for_job(job_path) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] error_desc=err_desc) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. 2021-09-10 16:57:05.411 4076 3051112034376 MainThread TRACE nova.compute.manager [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.compute.utils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Build of instance 8636793a-b94f-447d-a53a-4506e8a3032f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000031' failed to start. Not enough memory in the system to start the virtual machine instance-00000031. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000031' failed to start. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Not enough memory in the system to start the virtual machine instance-00000031 with ram size 1024 megabytes. (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) 'instance-00000031' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 292D1599-0940-4B99-A4FA-04DFB11EFCA5) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:57:05.411 4076 3051112034376 MainThread DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:57:05.724 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': 'E22717DA-B35D-48DE-A486-76CE66EBC13F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14815), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:05.856 4076 3051112036488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:05.856 4076 3051112036488 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:05.856 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:05.856 4076 3051112035608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:05.856 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E22717DA-B35D-48DE-A486-76CE66EBC13F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111318), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:05.856 4076 3051112036136 MainThread DEBUG os_win._utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5173628330230713 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:05.888 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-ff38537a-20b6-4460-b3ad-c094f1f40f93 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:05.888 4076 3051112036488 MainThread DEBUG nova.virt.hyperv.vmops [req-ff38537a-20b6-4460-b3ad-c094f1f40f93 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:05.948 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-ff38537a-20b6-4460-b3ad-c094f1f40f93 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:05.973 4076 3051112036488 MainThread INFO nova.compute.manager [req-ff38537a-20b6-4460-b3ad-c094f1f40f93 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:06.176 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-1c70ef04-141c-4d04-960a-01f60c7c5a52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Received event network-vif-plugged-644e3ea2-6bb8-448d-91e3-121af5fcdfa2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:06.176 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-1c70ef04-141c-4d04-960a-01f60c7c5a52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:06.192 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [req-1c70ef04-141c-4d04-960a-01f60c7c5a52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:06.192 4076 3051112036488 MainThread DEBUG nova.compute.manager [req-1c70ef04-141c-4d04-960a-01f60c7c5a52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Processing event network-vif-plugged-644e3ea2-6bb8-448d-91e3-121af5fcdfa2 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:57:06.225 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Using config drive for instance 2021-09-10 16:57:06.634 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Creating config drive at C:\openstack\instances\instance-00000033\configdrive.iso 2021-09-10 16:57:06.676 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000033\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu5l4e9kh execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:57:06.880 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': '3B01CB79-A7CC-456C-8488-6076CBBB3140', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20646), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:06.986 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000033\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpu5l4e9kh" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:57:07.014 4076 3051112035960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:07.014 4076 3051112035960 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:07.014 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:07.014 4076 3051112036488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:07.045 4076 3051112035960 MainThread DEBUG nova.compute.manager [req-3d667921-a3f5-43a6-b1cd-6fcf3bc6d06d - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:07.045 4076 3051112035960 MainThread DEBUG nova.virt.hyperv.vmops [req-3d667921-a3f5-43a6-b1cd-6fcf3bc6d06d - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:07.045 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3B01CB79-A7CC-456C-8488-6076CBBB3140', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113885), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:07.056 4076 3051112036136 MainThread DEBUG os_win._utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7170119285583496 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:07.092 4076 3051112035960 MainThread DEBUG nova.compute.manager [req-3d667921-a3f5-43a6-b1cd-6fcf3bc6d06d - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:07.130 4076 3051112035960 MainThread INFO nova.compute.manager [req-3d667921-a3f5-43a6-b1cd-6fcf3bc6d06d - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:07.223 4076 3051112034376 MainThread DEBUG nova.network.neutron [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:07.239 4076 3051112034376 MainThread INFO nova.compute.manager [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] [instance: 8636793a-b94f-447d-a53a-4506e8a3032f] Took 1.83 seconds to deallocate network for instance. 2021-09-10 16:57:07.400 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Setting boot order for instance: instance-00000033: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:57:07.437 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:57:07.473 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '0462F907-F7AB-4E29-9611-C92AEF8EA056', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16678), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:07.520 4076 3051112034376 MainThread INFO nova.scheduler.client.report [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Deleted allocations for instance 8636793a-b94f-447d-a53a-4506e8a3032f 2021-09-10 16:57:07.571 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [req-3330bb93-aead-4609-abf4-e7f1b2893ccb 52b9f83ffb604b3388b2c32b65a2e8ff e36daecbd54a4138802bd46bc59ade0d - default default] Lock "8636793a-b94f-447d-a53a-4506e8a3032f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 21.172s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:07.611 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:07.611 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:07.621 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:07.621 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:07.621 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0462F907-F7AB-4E29-9611-C92AEF8EA056', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114306), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:07.621 4076 3051113162360 MainThread DEBUG os_win._utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18443965911865234 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:07.645 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-a23446a6-2eb7-40c7-a9ad-80cbf366558e - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:07.645 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-a23446a6-2eb7-40c7-a9ad-80cbf366558e - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:07.691 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-a23446a6-2eb7-40c7-a9ad-80cbf366558e - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:07.754 4076 3051053528872 MainThread INFO nova.compute.manager [req-a23446a6-2eb7-40c7-a9ad-80cbf366558e - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:08.082 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': '8EE3C273-8E47-42D7-A522-EB027DA5B72B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14654), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:08.208 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:08.208 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:08.208 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:08.208 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:08.238 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-ba10c358-d04a-42d9-bc3a-c5cf59d03b64 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:08.238 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-ba10c358-d04a-42d9-bc3a-c5cf59d03b64 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:08.238 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8EE3C273-8E47-42D7-A522-EB027DA5B72B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110700), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:08.254 4076 3051112036136 MainThread DEBUG os_win._utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.915446043014526 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:08.275 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-ba10c358-d04a-42d9-bc3a-c5cf59d03b64 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:08.321 4076 3051053528872 MainThread INFO nova.compute.manager [req-ba10c358-d04a-42d9-bc3a-c5cf59d03b64 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:08.661 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '8F4C474B-768A-4969-BBC0-E459EC8F600E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14670), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:08.784 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:08.784 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:08.797 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:08.797 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:08.797 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-54a2ad92-d8e1-44da-a1b3-2e94d521c8cc - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:08.797 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-54a2ad92-d8e1-44da-a1b3-2e94d521c8cc - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:08.821 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8F4C474B-768A-4969-BBC0-E459EC8F600E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112470), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:08.821 4076 3051113162360 MainThread DEBUG os_win._utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3846056461334229 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:08.864 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-54a2ad92-d8e1-44da-a1b3-2e94d521c8cc - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:08.879 4076 3051053528872 MainThread INFO nova.compute.manager [req-54a2ad92-d8e1-44da-a1b3-2e94d521c8cc - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:09.286 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': '423CEA9E-E875-49F5-ADA8-40003E327784', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15461), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:09.416 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:09.416 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:09.416 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:09.416 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] VM Stopped (Lifecycle Event) 2021-09-10 16:57:09.416 4076 3051112036136 MainThread ERROR os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000032\' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE6110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000032\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)instance-0000003265E399EC-C049-410D-8BBC-54C4DE88CE61%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '423CEA9E-E875-49F5-ADA8-40003E327784', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114114), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\nNot enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)\n\n'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61)", 'ErrorSummaryDescription': "'instance-00000032' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000032.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:09.416 4076 3051112036136 MainThread ERROR nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Failed to change vm state of instance-00000032 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. 2021-09-10 16:57:09.416 4076 3051112036136 MainThread INFO nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Got request to destroy instance 2021-09-10 16:57:09.458 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-7f524772-23b0-4e55-af33-43ee01469ab9 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:09.458 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-7f524772-23b0-4e55-af33-43ee01469ab9 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:09.503 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-7f524772-23b0-4e55-af33-43ee01469ab9 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:09.535 4076 3051053528872 MainThread INFO nova.compute.manager [req-7f524772-23b0-4e55-af33-43ee01469ab9 - - - - -] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:09.844 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'E2DD3F55-E8FA-4DCE-B524-0046515EEFBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14664), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:09.996 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E2DD3F55-E8FA-4DCE-B524-0046515EEFBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113252), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:09.996 4076 3051113162360 MainThread DEBUG os_win._utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5590944290161133 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:09.996 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:09.996 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:09.996 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:09.996 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:10.027 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-acbe0903-37d3-4e41-9950-9ee8305626a0 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:10.027 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-acbe0903-37d3-4e41-9950-9ee8305626a0 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:10.066 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-acbe0903-37d3-4e41-9950-9ee8305626a0 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:10.114 4076 3051053528872 MainThread INFO nova.compute.manager [req-acbe0903-37d3-4e41-9950-9ee8305626a0 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:11.024 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'B57E05B7-F8A8-40BC-A40D-8B99C8CAFA93', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15274), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:11.158 4076 3051053528872 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:11.158 4076 3051053528872 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:11.158 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:11.158 4076 3051112034376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:11.158 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B57E05B7-F8A8-40BC-A40D-8B99C8CAFA93', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113865), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:11.171 4076 3051113162360 MainThread DEBUG os_win._utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7348227500915527 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:11.171 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-9c592ad2-83cf-4cb9-8391-4985e58a9d79 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:11.188 4076 3051053528872 MainThread DEBUG nova.virt.hyperv.vmops [req-9c592ad2-83cf-4cb9-8391-4985e58a9d79 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:11.229 4076 3051053528872 MainThread DEBUG nova.compute.manager [req-9c592ad2-83cf-4cb9-8391-4985e58a9d79 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:11.262 4076 3051053528872 MainThread INFO nova.compute.manager [req-9c592ad2-83cf-4cb9-8391-4985e58a9d79 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:11.262 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:57:11.262 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:11.262 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:11.286 4076 3051112036136 MainThread DEBUG os_win.utils.compute.vmutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Successfully changed vm state of instance-00000032 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:57:11.286 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Successfully changed state of VM instance-00000032 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:57:11.337 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Waiting for WMI job: {'InstanceID': '9D47FCAD-E5A5-4E0B-B5EB-920E9BDCB504', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14771), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:11.441 4076 3051112036136 MainThread DEBUG os_win.utils.jobutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] WMI job succeeded: {'InstanceID': '9D47FCAD-E5A5-4E0B-B5EB-920E9BDCB504', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29508), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:57:11.441 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:11.441 4076 3051112036136 MainThread DEBUG os_win.utils.pathutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Removing directory: C:\openstack\instances\instance-00000032 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:57:11.461 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:11.461 4076 3051112036136 MainThread ERROR nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Traceback (most recent call last): 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(*args, **kwargs) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] admin_password, network_info, block_device_info) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(self, *args, **kwds) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] should_plug_vifs=False) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'req_state': req_state}) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] time.sleep(sleep_time) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return f(*args, **kwargs) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return self._wait_for_job(job_path) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] error_desc=err_desc) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During handling of the above exception, another exception occurred: 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Traceback (most recent call last): 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] yield resources 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] accel_info=accel_info) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(*args, **kwargs) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] admin_password, network_info, block_device_info) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(self, *args, **kwds) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] should_plug_vifs=False) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'req_state': req_state}) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] time.sleep(sleep_time) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return f(*args, **kwargs) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return self._wait_for_job(job_path) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] error_desc=err_desc) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.461 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.461 4076 3051112036136 MainThread INFO nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Terminating instance 2021-09-10 16:57:11.461 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:57:11.461 4076 3051112036136 MainThread INFO nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Got request to destroy instance 2021-09-10 16:57:11.477 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:57:11.477 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:11.477 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "instance-00000032" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:11.477 4076 3051112036136 MainThread INFO nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:57:11.477 4076 3051112036136 MainThread DEBUG nova.compute.claims [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:57:11.477 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:11.668 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.hostops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:11.684 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.hostops [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:11.894 4076 3051112036136 MainThread DEBUG nova.compute.provider_tree [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.scheduler.client.report [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:11.918 4076 3051112036136 MainThread ERROR nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Traceback (most recent call last): 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(*args, **kwargs) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] admin_password, network_info, block_device_info) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(self, *args, **kwds) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] should_plug_vifs=False) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'req_state': req_state}) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] time.sleep(sleep_time) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return f(*args, **kwargs) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return self._wait_for_job(job_path) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] error_desc=err_desc) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] During handling of the above exception, another exception occurred: 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Traceback (most recent call last): 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] accel_info=accel_info) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(*args, **kwargs) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] admin_password, network_info, block_device_info) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return function(self, *args, **kwds) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] should_plug_vifs=False) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'req_state': req_state}) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] time.sleep(sleep_time) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self.force_reraise() 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] raise self.value 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return f(*args, **kwargs) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] return self._wait_for_job(job_path) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] error_desc=err_desc) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. 2021-09-10 16:57:11.918 4076 3051112036136 MainThread TRACE nova.compute.manager [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.compute.utils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Build of instance cf1f3ab0-f978-4556-a81a-1be7a5bbfd12 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000032' failed to start. Not enough memory in the system to start the virtual machine instance-00000032. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000032' failed to start. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Not enough memory in the system to start the virtual machine instance-00000032 with ram size 1024 megabytes. (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) 'instance-00000032' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 65E399EC-C049-410D-8BBC-54C4DE88CE61) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:57:11.918 4076 3051112036136 MainThread DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:57:12.192 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '5315CF97-6FDA-4BBE-9D0E-B1F9D054BCEC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15230), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:12.325 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:12.325 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:12.325 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:12.325 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:12.340 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5315CF97-6FDA-4BBE-9D0E-B1F9D054BCEC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110259), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:12.340 4076 3051113162360 MainThread DEBUG os_win._utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.903066158294678 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:12.340 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-5c630912-76db-4533-9b48-8333d6d023a7 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:12.340 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-5c630912-76db-4533-9b48-8333d6d023a7 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:12.399 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-5c630912-76db-4533-9b48-8333d6d023a7 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:12.430 4076 3051113163416 MainThread INFO nova.compute.manager [req-5c630912-76db-4533-9b48-8333d6d023a7 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:12.519 4076 3051112036136 MainThread DEBUG nova.network.neutron [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:12.528 4076 3051112036136 MainThread INFO nova.compute.manager [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] [instance: cf1f3ab0-f978-4556-a81a-1be7a5bbfd12] Took 0.61 seconds to deallocate network for instance. 2021-09-10 16:57:12.848 4076 3051112036136 MainThread INFO nova.scheduler.client.report [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Deleted allocations for instance cf1f3ab0-f978-4556-a81a-1be7a5bbfd12 2021-09-10 16:57:12.910 4076 3051112036136 MainThread DEBUG oslo_concurrency.lockutils [req-425c8b52-7152-4944-86e2-3e9c56157ea1 4b444bbec44249dda082cd505f5a9c27 2902d375c2104912b7ea540517edd89f - default default] Lock "cf1f3ab0-f978-4556-a81a-1be7a5bbfd12" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.578s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:13.353 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'DF1170CB-56F6-42F4-9DD6-076704B9303E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18627), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:13.480 4076 3051112036136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:13.480 4076 3051112036136 MainThread INFO nova.compute.manager [-] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] VM Stopped (Lifecycle Event) 2021-09-10 16:57:13.504 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:13.504 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:13.504 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000033\' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000033\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)instance-00000033E1B7722C-9433-4930-945A-15B8082CBFC0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DF1170CB-56F6-42F4-9DD6-076704B9303E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112172), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\nNot enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)\n\n'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0)", 'ErrorSummaryDescription': "'instance-00000033' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000033.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:13.504 4076 3051113162360 MainThread ERROR nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Failed to change vm state of instance-00000033 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. 2021-09-10 16:57:13.504 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Got request to destroy instance 2021-09-10 16:57:13.524 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-f13e6c93-6e29-417d-b549-017dc34cc165 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:13.524 4076 3051112036136 MainThread DEBUG nova.virt.hyperv.vmops [req-f13e6c93-6e29-417d-b549-017dc34cc165 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:13.573 4076 3051112036136 MainThread DEBUG nova.compute.manager [req-f13e6c93-6e29-417d-b549-017dc34cc165 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:13.618 4076 3051112036136 MainThread INFO nova.compute.manager [req-f13e6c93-6e29-417d-b549-017dc34cc165 - - - - -] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:15.319 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:57:15.319 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:15.332 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:15.332 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed vm state of instance-00000033 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:57:15.332 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed state of VM instance-00000033 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:57:15.374 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '7C3CF53B-69F1-401A-97F8-AA911646865A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14839), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:15.489 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job succeeded: {'InstanceID': '7C3CF53B-69F1-401A-97F8-AA911646865A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28326), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:57:15.489 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:15.489 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Removing directory: C:\openstack\instances\instance-00000033 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:57:15.489 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:15.489 4076 3051113162360 MainThread ERROR nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Traceback (most recent call last): 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(*args, **kwargs) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] admin_password, network_info, block_device_info) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(self, *args, **kwds) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] should_plug_vifs=False) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'req_state': req_state}) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] time.sleep(sleep_time) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return f(*args, **kwargs) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return self._wait_for_job(job_path) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] error_desc=err_desc) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During handling of the above exception, another exception occurred: 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Traceback (most recent call last): 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] yield resources 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] accel_info=accel_info) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(*args, **kwargs) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] admin_password, network_info, block_device_info) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(self, *args, **kwds) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] should_plug_vifs=False) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'req_state': req_state}) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] time.sleep(sleep_time) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return f(*args, **kwargs) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return self._wait_for_job(job_path) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] error_desc=err_desc) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.489 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.489 4076 3051113162360 MainThread INFO nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Terminating instance 2021-09-10 16:57:15.489 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:57:15.489 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Got request to destroy instance 2021-09-10 16:57:15.504 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:57:15.504 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:15.504 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000033" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:15.519 4076 3051113162360 MainThread INFO nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:57:15.519 4076 3051113162360 MainThread DEBUG nova.compute.claims [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:57:15.519 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:15.653 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:15.676 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:15.894 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:15.904 4076 3051113162360 MainThread ERROR nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Traceback (most recent call last): 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(*args, **kwargs) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] admin_password, network_info, block_device_info) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(self, *args, **kwds) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] should_plug_vifs=False) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'req_state': req_state}) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] time.sleep(sleep_time) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return f(*args, **kwargs) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return self._wait_for_job(job_path) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] error_desc=err_desc) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] During handling of the above exception, another exception occurred: 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Traceback (most recent call last): 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] accel_info=accel_info) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(*args, **kwargs) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] admin_password, network_info, block_device_info) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return function(self, *args, **kwds) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] should_plug_vifs=False) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'req_state': req_state}) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] time.sleep(sleep_time) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self.force_reraise() 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] raise self.value 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return f(*args, **kwargs) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] return self._wait_for_job(job_path) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] error_desc=err_desc) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. 2021-09-10 16:57:15.904 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Build of instance 19bd864f-b01d-42d3-8ca6-9b75067146fa was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000033' failed to start. Not enough memory in the system to start the virtual machine instance-00000033. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000033' failed to start. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Not enough memory in the system to start the virtual machine instance-00000033 with ram size 1024 megabytes. (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) 'instance-00000033' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E1B7722C-9433-4930-945A-15B8082CBFC0) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:57:15.904 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:57:15.924 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:57:15.924 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:57:15.993 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.114 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.128 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 16:57:16.128 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113164296 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113164296 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113292200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292200 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292552 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113292904 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113292904 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.128 4076 3051113291848 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.128 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.144 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.144 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:16.144 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:16.250 4076 3051113164296 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.110s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.286 4076 3051113293080 MainThread WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 11, greater than warning threshold: 10. There could be a leak. Increasing threshold to: 20 2021-09-10 16:57:16.286 4076 3051113292200 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.337 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.400 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.424 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.441 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.473 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.479 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.520 4076 3051113292904 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.528 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.528 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:16.544 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:20.432 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:20.461 4076 3051113162360 MainThread INFO nova.compute.manager [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 19bd864f-b01d-42d3-8ca6-9b75067146fa] Took 4.53 seconds to deallocate network for instance. 2021-09-10 16:57:20.863 4076 3051113162360 MainThread INFO nova.scheduler.client.report [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Deleted allocations for instance 19bd864f-b01d-42d3-8ca6-9b75067146fa 2021-09-10 16:57:20.911 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-3e4a9c84-ec4f-47c2-b602-881c1f0b63bb 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "19bd864f-b01d-42d3-8ca6-9b75067146fa" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.968s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:27.003 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:29.821 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:29.821 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:57:29.821 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 16:57:29.988 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:29.988 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:57:29.988 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 16:57:30.914 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:30.941 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:30.941 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:57:30.941 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:31.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:31.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:31.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:32.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:33.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:33.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:33.809 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:33.809 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:33.809 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:57:33.809 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:33.837 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:34.059 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2208MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:57:34.059 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:34.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 INFO nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a60acc40-a373-4430-b931-0d33ee511d49 has allocations against this compute host but is not found in the database. 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.410 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:57:34.410 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:57:34.410 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:57:34.431 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:34.457 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:34.659 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:34.681 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:34.681 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:57:34.681 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:34.738 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:34.769 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:57:34.957 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:34.957 4076 3051113162360 MainThread DEBUG nova.virt.hardware [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:57:34.957 4076 3051113162360 MainThread INFO nova.compute.claims [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Claim successful on node n-h1-808474-1 2021-09-10 16:57:35.128 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:35.144 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:35.337 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:35.378 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:35.378 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:35.378 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:57:35.449 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:57:35.449 4076 3051113164472 GreenThread-39 DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:57:35.449 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:57:35.488 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:57:35.691 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:57:35.691 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Spawning new instance 2021-09-10 16:57:35.706 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG nova.block_device [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Creating directory: C:\openstack\instances\instance-00000034 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:57:35.712 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000034\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:57:35.747 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Resizing VHD C:\openstack\instances\instance-00000034\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:57:35.883 4076 3051113164472 GreenThread-39 DEBUG nova.policy [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3e3974028a49412ba2f9310b9ea9829a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3a6913db09dc4c04925b36d6424e7e89', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:57:37.675 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:57:37.675 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:57:38.672 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Successfully created port: 2d612da8-570b-4bfc-b9d0-26ea993617ea _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:57:40.504 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Successfully updated port: 2d612da8-570b-4bfc-b9d0-26ea993617ea _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:57:40.534 4076 3051113164472 GreenThread-39 DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Acquired lock "refresh_cache-a60acc40-a373-4430-b931-0d33ee511d49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:40.534 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:57:40.581 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Received event network-changed-2d612da8-570b-4bfc-b9d0-26ea993617ea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:40.581 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Refreshing instance network info cache due to event network-changed-2d612da8-570b-4bfc-b9d0-26ea993617ea. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:57:40.607 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:57:41.176 4076 3051113164472 GreenThread-39 DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Updating instance_info_cache with network_info: [{"id": "2d612da8-570b-4bfc-b9d0-26ea993617ea", "address": "fa:16:3e:65:97:25", "network": {"id": "b73e1d6e-6c44-444d-ac7e-cc92dc83874a", "bridge": null, "label": "tempest-ServerAddressesTestJSON-1219095971-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3a6913db09dc4c04925b36d6424e7e89", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2d612da8-57", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:41.206 4076 3051113164472 GreenThread-39 DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Releasing lock "refresh_cache-a60acc40-a373-4430-b931-0d33ee511d49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:41.206 4076 3051113164472 GreenThread-39 DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Instance network_info: |[{"id": "2d612da8-570b-4bfc-b9d0-26ea993617ea", "address": "fa:16:3e:65:97:25", "network": {"id": "b73e1d6e-6c44-444d-ac7e-cc92dc83874a", "bridge": null, "label": "tempest-ServerAddressesTestJSON-1219095971-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3a6913db09dc4c04925b36d6424e7e89", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2d612da8-57", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:57:41.222 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a60acc40-a373-4430-b931-0d33ee511d49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:41.222 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Refreshing network info cache for port 2d612da8-570b-4bfc-b9d0-26ea993617ea _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:57:41.222 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Preparing to wait for external event network-vif-plugged-2d612da8-570b-4bfc-b9d0-26ea993617ea prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:57:41.222 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:41.222 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:41.222 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Creating VM instance-00000034 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:57:42.071 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:57:42.767 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Updated VIF entry in instance network info cache for port 2d612da8-570b-4bfc-b9d0-26ea993617ea. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:57:42.767 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Updating instance_info_cache with network_info: [{"id": "2d612da8-570b-4bfc-b9d0-26ea993617ea", "address": "fa:16:3e:65:97:25", "network": {"id": "b73e1d6e-6c44-444d-ac7e-cc92dc83874a", "bridge": null, "label": "tempest-ServerAddressesTestJSON-1219095971-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3a6913db09dc4c04925b36d6424e7e89", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2d612da8-57", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:42.807 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-c3af3e65-39e3-403c-9b73-613b22e34472 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a60acc40-a373-4430-b931-0d33ee511d49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:47.015 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:47.031 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:57:47.207 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:47.207 4076 3051113164120 MainThread DEBUG nova.virt.hardware [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:57:47.207 4076 3051113164120 MainThread INFO nova.compute.claims [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Claim successful on node n-h1-808474-1 2021-09-10 16:57:47.367 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:47.388 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:47.581 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:47.597 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:47.597 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:47.597 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:57:47.707 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:57:47.707 4076 3051113164648 GreenThread-40 DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:57:47.707 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:57:47.722 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:57:47.742 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9a84fa45-9c90-430a-895d-3cda106db7bf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Received event network-vif-plugged-2d612da8-570b-4bfc-b9d0-26ea993617ea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:47.742 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-9a84fa45-9c90-430a-895d-3cda106db7bf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:47.742 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-9a84fa45-9c90-430a-895d-3cda106db7bf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:47.742 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9a84fa45-9c90-430a-895d-3cda106db7bf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Processing event network-vif-plugged-2d612da8-570b-4bfc-b9d0-26ea993617ea _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:57:47.829 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Using config drive for instance 2021-09-10 16:57:47.940 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:57:47.956 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Spawning new instance 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG nova.block_device [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Creating directory: C:\openstack\instances\instance-00000035 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:57:47.971 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000035\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:57:48.008 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Resizing VHD C:\openstack\instances\instance-00000035\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:57:48.128 4076 3051113164648 GreenThread-40 DEBUG nova.policy [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5caa9c73de9d44b4a4001f521e76cb24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6153c0b18ada42b8b1fc8a04fbd01d24', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:57:48.394 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Creating config drive at C:\openstack\instances\instance-00000034\configdrive.iso 2021-09-10 16:57:48.430 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000034\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp0ncptvr9 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:57:48.754 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000034\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp0ncptvr9" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:57:49.262 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Setting boot order for instance: instance-00000034: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:57:49.309 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:57:49.345 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': 'B3C0EFAB-7405-4E35-97FB-02AD9B25A226', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19228), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:49.479 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:49.479 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:49.479 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:49.479 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:49.508 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B3C0EFAB-7405-4E35-97FB-02AD9B25A226', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113085), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:49.508 4076 3051113162360 MainThread DEBUG os_win._utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1985468864440918 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:49.519 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-86647d57-058e-431c-b264-61e62acc235e - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:49.519 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-86647d57-058e-431c-b264-61e62acc235e - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:49.565 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-86647d57-058e-431c-b264-61e62acc235e - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:49.581 4076 3051113163416 MainThread INFO nova.compute.manager [req-86647d57-058e-431c-b264-61e62acc235e - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:50.035 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Successfully created port: 6aeeeab6-03c6-481f-9560-04922ab36f30 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:57:50.532 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': '92DCAFE4-384F-4372-8A6E-0075A776AA29', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14436), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:50.651 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:50.651 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:50.651 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:50.651 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:50.686 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '92DCAFE4-384F-4372-8A6E-0075A776AA29', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110337), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:50.686 4076 3051113162360 MainThread DEBUG os_win._utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3768668174743652 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:50.686 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-fb0a18bd-fab8-420e-ab61-47b14e36c3ff - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:50.686 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-fb0a18bd-fab8-420e-ab61-47b14e36c3ff - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:50.736 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-fb0a18bd-fab8-420e-ab61-47b14e36c3ff - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:50.789 4076 3051113163416 MainThread INFO nova.compute.manager [req-fb0a18bd-fab8-420e-ab61-47b14e36c3ff - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:51.703 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': 'C5795B44-E8CD-4805-8E53-56D06A4F33E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15455), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:51.831 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:51.831 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:51.831 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:51.831 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:51.831 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C5795B44-E8CD-4805-8E53-56D06A4F33E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109789), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:51.831 4076 3051113162360 MainThread DEBUG os_win._utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5221643447875977 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:51.878 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-e56a4c52-12d2-459f-905e-bba76ea0241a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:51.878 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-e56a4c52-12d2-459f-905e-bba76ea0241a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:51.925 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-e56a4c52-12d2-459f-905e-bba76ea0241a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:51.972 4076 3051113163416 MainThread INFO nova.compute.manager [req-e56a4c52-12d2-459f-905e-bba76ea0241a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:52.285 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Successfully updated port: 6aeeeab6-03c6-481f-9560-04922ab36f30 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:57:52.316 4076 3051113164648 GreenThread-40 DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Acquired lock "refresh_cache-0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:52.316 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:57:52.581 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:57:52.867 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': '90F02836-F922-4C1A-B0FA-AD297BFAE0AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15920), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:52.978 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:52.978 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:52.978 4076 3051113164472 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:52.978 4076 3051113164472 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:53.008 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '90F02836-F922-4C1A-B0FA-AD297BFAE0AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110561), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:53.008 4076 3051113162360 MainThread DEBUG os_win._utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.698676109313965 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:53.008 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-9b598529-1168-4547-a910-f40838259d49 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:53.008 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [req-9b598529-1168-4547-a910-f40838259d49 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:53.064 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-9b598529-1168-4547-a910-f40838259d49 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:53.087 4076 3051113164472 MainThread INFO nova.compute.manager [req-9b598529-1168-4547-a910-f40838259d49 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:53.429 4076 3051113164648 GreenThread-40 DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Updating instance_info_cache with network_info: [{"id": "6aeeeab6-03c6-481f-9560-04922ab36f30", "address": "fa:16:3e:bf:34:0e", "network": {"id": "757f8fb5-ff7d-411a-968b-dea6648df5e2", "bridge": null, "label": "tempest-VolumesAdminNegativeTest-41234616-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6153c0b18ada42b8b1fc8a04fbd01d24", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6aeeeab6-03", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:53.458 4076 3051113164648 GreenThread-40 DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Releasing lock "refresh_cache-0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:53.458 4076 3051113164648 GreenThread-40 DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Instance network_info: |[{"id": "6aeeeab6-03c6-481f-9560-04922ab36f30", "address": "fa:16:3e:bf:34:0e", "network": {"id": "757f8fb5-ff7d-411a-968b-dea6648df5e2", "bridge": null, "label": "tempest-VolumesAdminNegativeTest-41234616-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6153c0b18ada42b8b1fc8a04fbd01d24", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6aeeeab6-03", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:57:53.458 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Preparing to wait for external event network-vif-plugged-6aeeeab6-03c6-481f-9560-04922ab36f30 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:57:53.458 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:53.458 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:53.458 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Creating VM instance-00000035 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:57:53.941 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Received event network-changed-6aeeeab6-03c6-481f-9560-04922ab36f30 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:57:53.941 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Refreshing instance network info cache due to event network-changed-6aeeeab6-03c6-481f-9560-04922ab36f30. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:57:53.941 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:57:53.941 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Refreshing network info cache for port 6aeeeab6-03c6-481f-9560-04922ab36f30 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:57:54.027 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': 'C6DBCFC8-33AB-4026-9B39-C1415ACE2A97', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15232), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:54.164 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C6DBCFC8-33AB-4026-9B39-C1415ACE2A97', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108918), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:54.164 4076 3051113162360 MainThread DEBUG os_win._utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.854785442352295 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:57:54.164 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:54.164 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:54.164 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:54.164 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:54.184 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ef8fadd2-72e3-47a5-955e-85cf2bbd0d6a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:54.184 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ef8fadd2-72e3-47a5-955e-85cf2bbd0d6a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:54.244 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ef8fadd2-72e3-47a5-955e-85cf2bbd0d6a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:54.304 4076 3051113164648 MainThread INFO nova.compute.manager [req-ef8fadd2-72e3-47a5-955e-85cf2bbd0d6a - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:54.363 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:57:54.751 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:54.775 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:57:54.887 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:54.910 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:57:54.910 4076 3051113164648 MainThread INFO nova.compute.claims [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Claim successful on node n-h1-808474-1 2021-09-10 16:57:55.055 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:55.055 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:55.211 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': '636B47F4-88B9-4211-9857-9500A1419E3C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15056), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:55.279 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:55.307 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:55.307 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:55.307 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:57:55.338 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:55.338 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:55.338 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:57:55.338 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: a60acc40-a373-4430-b931-0d33ee511d49] VM Stopped (Lifecycle Event) 2021-09-10 16:57:55.356 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000034\' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000034\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)instance-0000003415B60522-B458-4DB6-85B7-0C868F5D7EEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '636B47F4-88B9-4211-9857-9500A1419E3C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 57, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 57, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\nNot enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)\n\n'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB)", 'ErrorSummaryDescription': "'instance-00000034' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000034.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:57:55.356 4076 3051113162360 MainThread ERROR nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Failed to change vm state of instance-00000034 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. 2021-09-10 16:57:55.356 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Got request to destroy instance 2021-09-10 16:57:55.372 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-f0349f29-9d71-4692-b2f1-7ff63b0e56c0 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:57:55.372 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-f0349f29-9d71-4692-b2f1-7ff63b0e56c0 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:57:55.406 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-f0349f29-9d71-4692-b2f1-7ff63b0e56c0 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:57:55.406 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:57:55.430 4076 3051113162536 GreenThread-41 DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:57:55.430 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:57:55.476 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:57:55.476 4076 3051113163416 MainThread INFO nova.compute.manager [req-f0349f29-9d71-4692-b2f1-7ff63b0e56c0 - - - - -] [instance: a60acc40-a373-4430-b931-0d33ee511d49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:57:55.681 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:57:55.681 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Spawning new instance 2021-09-10 16:57:55.699 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:55.699 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:55.699 4076 3051113164648 MainThread DEBUG nova.block_device [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:57:55.699 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:55.699 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:55.712 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating directory: C:\openstack\instances\instance-00000036 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:57:55.712 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000036\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:57:55.744 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Resizing VHD C:\openstack\instances\instance-00000036\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:57:55.918 4076 3051113162536 GreenThread-41 DEBUG nova.policy [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '42267190de3648eea52604e6b2444d20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6ca07d9dc99a4581a698a232ffa065db', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:57:55.961 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Updated VIF entry in instance network info cache for port 6aeeeab6-03c6-481f-9560-04922ab36f30. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:57:55.961 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Updating instance_info_cache with network_info: [{"id": "6aeeeab6-03c6-481f-9560-04922ab36f30", "address": "fa:16:3e:bf:34:0e", "network": {"id": "757f8fb5-ff7d-411a-968b-dea6648df5e2", "bridge": null, "label": "tempest-VolumesAdminNegativeTest-41234616-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6153c0b18ada42b8b1fc8a04fbd01d24", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6aeeeab6-03", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:57:55.961 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-f7a7b5a6-57fd-4d1b-9e8d-acc7c577010b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:57:57.202 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:57:57.202 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:57.202 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:57.217 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Successfully changed vm state of instance-00000034 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:57:57.217 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Successfully changed state of VM instance-00000034 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:57:57.261 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Waiting for WMI job: {'InstanceID': '55BA93C1-FE89-458C-B0AA-8B3F5FF58641', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14176), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:57:57.394 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] WMI job succeeded: {'InstanceID': '55BA93C1-FE89-458C-B0AA-8B3F5FF58641', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29227), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:57:57.394 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:57.394 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Removing directory: C:\openstack\instances\instance-00000034 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:57:57.394 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:57.394 4076 3051113162360 MainThread ERROR nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Traceback (most recent call last): 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(*args, **kwargs) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] admin_password, network_info, block_device_info) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(self, *args, **kwds) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] should_plug_vifs=False) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'req_state': req_state}) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] time.sleep(sleep_time) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return f(*args, **kwargs) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return self._wait_for_job(job_path) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] error_desc=err_desc) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] During handling of the above exception, another exception occurred: 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Traceback (most recent call last): 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] yield resources 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] accel_info=accel_info) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(*args, **kwargs) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] admin_password, network_info, block_device_info) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(self, *args, **kwds) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] should_plug_vifs=False) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'req_state': req_state}) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] time.sleep(sleep_time) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return f(*args, **kwargs) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return self._wait_for_job(job_path) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] error_desc=err_desc) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.394 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.394 4076 3051113162360 MainThread INFO nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Terminating instance 2021-09-10 16:57:57.394 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:57:57.394 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Got request to destroy instance 2021-09-10 16:57:57.414 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:57:57.414 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:57.414 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "instance-00000034" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:57.427 4076 3051113162360 MainThread INFO nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:57:57.427 4076 3051113162360 MainThread DEBUG nova.compute.claims [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:57:57.427 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:57.539 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:57.559 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:57:57.749 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:57:57.785 4076 3051113162360 MainThread ERROR nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Traceback (most recent call last): 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(*args, **kwargs) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] admin_password, network_info, block_device_info) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(self, *args, **kwds) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] should_plug_vifs=False) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'req_state': req_state}) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] time.sleep(sleep_time) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return f(*args, **kwargs) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return self._wait_for_job(job_path) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] error_desc=err_desc) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] During handling of the above exception, another exception occurred: 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Traceback (most recent call last): 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] accel_info=accel_info) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(*args, **kwargs) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] admin_password, network_info, block_device_info) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return function(self, *args, **kwds) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] should_plug_vifs=False) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'req_state': req_state}) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] time.sleep(sleep_time) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self.force_reraise() 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] raise self.value 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return f(*args, **kwargs) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] return self._wait_for_job(job_path) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] error_desc=err_desc) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. 2021-09-10 16:57:57.785 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: a60acc40-a373-4430-b931-0d33ee511d49] 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Build of instance a60acc40-a373-4430-b931-0d33ee511d49 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000034' failed to start. Not enough memory in the system to start the virtual machine instance-00000034. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000034' failed to start. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Not enough memory in the system to start the virtual machine instance-00000034 with ram size 1024 megabytes. (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) 'instance-00000034' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 15B60522-B458-4DB6-85B7-0C868F5D7EEB) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:57:57.785 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:57:59.566 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Successfully created port: 1e195453-3f7d-47ae-b9de-c2ba027332be _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:57:59.651 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:59.689 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:57:59.820 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:57:59.820 4076 3051113160952 MainThread DEBUG nova.virt.hardware [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:57:59.820 4076 3051113160952 MainThread INFO nova.compute.claims [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Claim successful on node n-h1-808474-1 2021-09-10 16:57:59.950 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.hostops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:57:59.950 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.hostops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:00.150 4076 3051113160952 MainThread DEBUG nova.compute.provider_tree [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:00.179 4076 3051113160952 MainThread DEBUG nova.scheduler.client.report [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:00.192 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:00.192 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:00.285 4076 3051113160952 MainThread DEBUG nova.compute.utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:00.285 4076 3051113160776 GreenThread-42 DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:00.285 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:00.309 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:00.637 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:00.637 4076 3051113160952 MainThread INFO nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Spawning new instance 2021-09-10 16:58:00.655 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:00.655 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:00.655 4076 3051113160952 MainThread DEBUG nova.block_device [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:00.655 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:00.655 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:00.671 4076 3051113160952 MainThread DEBUG os_win.utils.pathutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating directory: C:\openstack\instances\instance-00000037 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:58:00.671 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000037\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:58:00.701 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Resizing VHD C:\openstack\instances\instance-00000037\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:58:00.713 4076 3051113160776 GreenThread-42 DEBUG nova.policy [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e9b8fdd82fe341dc8eeee9f6dea5f66f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cc8daf6ce3114d6aa694f16669c8ed20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:01.133 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:01.165 4076 3051113162360 MainThread INFO nova.compute.manager [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] [instance: a60acc40-a373-4430-b931-0d33ee511d49] Took 3.38 seconds to deallocate network for instance. 2021-09-10 16:58:01.484 4076 3051113162360 MainThread INFO nova.scheduler.client.report [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Deleted allocations for instance a60acc40-a373-4430-b931-0d33ee511d49 2021-09-10 16:58:01.535 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-48498552-4d44-4c01-a370-1246776f8031 3e3974028a49412ba2f9310b9ea9829a 3a6913db09dc4c04925b36d6424e7e89 - default default] Lock "a60acc40-a373-4430-b931-0d33ee511d49" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.796s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:01.991 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:02.027 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:58:02.221 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:02.221 4076 3051113163064 MainThread DEBUG nova.virt.hardware [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:58:02.221 4076 3051113163064 MainThread INFO nova.compute.claims [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Claim successful on node n-h1-808474-1 2021-09-10 16:58:02.385 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:02.385 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:02.589 4076 3051113163064 MainThread DEBUG nova.compute.provider_tree [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:02.617 4076 3051113163064 MainThread DEBUG nova.scheduler.client.report [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:02.617 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:02.617 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:02.742 4076 3051113163064 MainThread DEBUG nova.compute.utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:02.773 4076 3051113162360 GreenThread-43 DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:02.773 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:02.801 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:03.047 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:03.067 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Spawning new instance 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG nova.block_device [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG os_win.utils.pathutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating directory: C:\openstack\instances\instance-00000038 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:58:03.082 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000038\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:58:03.102 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Resizing VHD C:\openstack\instances\instance-00000038\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:58:03.544 4076 3051113162360 GreenThread-43 DEBUG nova.policy [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e9b8fdd82fe341dc8eeee9f6dea5f66f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cc8daf6ce3114d6aa694f16669c8ed20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:03.559 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Successfully updated port: 1e195453-3f7d-47ae-b9de-c2ba027332be _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:03.580 4076 3051113162536 GreenThread-41 DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Acquired lock "refresh_cache-4bdc3835-ba14-4e1b-bd49-23ec32f15291" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:03.580 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:03.679 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Successfully created port: 66ec8cb2-5a45-471f-a9e4-5e85bfaab509 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:03.992 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:04.684 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:04.730 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:58:04.792 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-9f49cf06-5598-4d4b-9691-a749029e98fa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Received event network-vif-plugged-6aeeeab6-03c6-481f-9560-04922ab36f30 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:04.792 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-9f49cf06-5598-4d4b-9691-a749029e98fa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:04.792 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-9f49cf06-5598-4d4b-9691-a749029e98fa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:04.792 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-9f49cf06-5598-4d4b-9691-a749029e98fa 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Processing event network-vif-plugged-6aeeeab6-03c6-481f-9560-04922ab36f30 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:04.854 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Using config drive for instance 2021-09-10 16:58:04.938 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:04.938 4076 3051113161656 MainThread DEBUG nova.virt.hardware [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:58:04.938 4076 3051113161656 MainThread INFO nova.compute.claims [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Claim successful on node n-h1-808474-1 2021-09-10 16:58:05.078 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.hostops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:05.096 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.hostops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:05.243 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Creating config drive at C:\openstack\instances\instance-00000035\configdrive.iso 2021-09-10 16:58:05.291 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000035\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp7pk2qqqs execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:05.430 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Received event network-changed-1e195453-3f7d-47ae-b9de-c2ba027332be external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:05.430 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Refreshing instance network info cache due to event network-changed-1e195453-3f7d-47ae-b9de-c2ba027332be. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:05.477 4076 3051113161656 MainThread DEBUG nova.compute.provider_tree [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:05.495 4076 3051113161656 MainThread DEBUG nova.scheduler.client.report [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:05.495 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:05.495 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:05.588 4076 3051113161656 MainThread DEBUG nova.compute.utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:05.588 4076 3051049643352 GreenThread-44 DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:05.588 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:05.602 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000035\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp7pk2qqqs" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:05.623 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:05.806 4076 3051113161656 MainThread INFO nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Spawning new instance 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG nova.block_device [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:05.806 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:05.832 4076 3051113161656 MainThread DEBUG os_win.utils.pathutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating directory: C:\openstack\instances\instance-00000039 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:58:05.832 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000039\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:58:05.851 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Resizing VHD C:\openstack\instances\instance-00000039\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:58:05.996 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Setting boot order for instance: instance-00000035: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:06.043 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:06.062 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': 'AE409C52-0A7A-4BC6-AC99-4E0CAF9C4AF7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14906), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:06.098 4076 3051049643352 GreenThread-44 DEBUG nova.policy [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e9b8fdd82fe341dc8eeee9f6dea5f66f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cc8daf6ce3114d6aa694f16669c8ed20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:06.202 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:06.202 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:06.228 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:06.228 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:06.228 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AE409C52-0A7A-4BC6-AC99-4E0CAF9C4AF7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113957), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:06.245 4076 3051113164120 MainThread DEBUG os_win._utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.20193767547607422 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:06.245 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-67da5c99-e543-4f9a-9a2d-cc0b5bbf5e99 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:06.245 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-67da5c99-e543-4f9a-9a2d-cc0b5bbf5e99 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:06.277 4076 3051113162536 GreenThread-41 DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Updating instance_info_cache with network_info: [{"id": "1e195453-3f7d-47ae-b9de-c2ba027332be", "address": "fa:16:3e:84:ef:aa", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e195453-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:06.295 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-67da5c99-e543-4f9a-9a2d-cc0b5bbf5e99 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:06.311 4076 3051113162536 GreenThread-41 DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Releasing lock "refresh_cache-4bdc3835-ba14-4e1b-bd49-23ec32f15291" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:06.311 4076 3051113162536 GreenThread-41 DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Instance network_info: |[{"id": "1e195453-3f7d-47ae-b9de-c2ba027332be", "address": "fa:16:3e:84:ef:aa", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e195453-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:06.311 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4bdc3835-ba14-4e1b-bd49-23ec32f15291" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:06.311 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Refreshing network info cache for port 1e195453-3f7d-47ae-b9de-c2ba027332be _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:06.311 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Preparing to wait for external event network-vif-plugged-1e195453-3f7d-47ae-b9de-c2ba027332be prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:06.321 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:06.321 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:06.321 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating VM instance-00000036 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:06.356 4076 3051113163768 MainThread INFO nova.compute.manager [req-67da5c99-e543-4f9a-9a2d-cc0b5bbf5e99 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:06.695 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Successfully updated port: 66ec8cb2-5a45-471f-a9e4-5e85bfaab509 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:06.712 4076 3051113160776 GreenThread-42 DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Acquired lock "refresh_cache-082d9c5d-01d4-43af-8493-80ba655501fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:06.712 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:06.949 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Received event network-changed-66ec8cb2-5a45-471f-a9e4-5e85bfaab509 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:06.949 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Refreshing instance network info cache due to event network-changed-66ec8cb2-5a45-471f-a9e4-5e85bfaab509. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:07.077 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:07.077 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Successfully created port: 7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:07.226 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:07.258 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': 'AAD7B11D-1568-4D05-9CA5-273B7E591342', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14651), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:07.404 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AAD7B11D-1568-4D05-9CA5-273B7E591342', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112318), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:07.404 4076 3051113164120 MainThread DEBUG os_win._utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3610329627990723 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:07.420 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:07.420 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:07.420 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:07.420 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:07.452 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-3b06ca0b-20d8-40ec-888e-5752f0e11348 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:07.452 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-3b06ca0b-20d8-40ec-888e-5752f0e11348 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:07.482 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-3b06ca0b-20d8-40ec-888e-5752f0e11348 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:07.520 4076 3051049643704 MainThread INFO nova.compute.manager [req-3b06ca0b-20d8-40ec-888e-5752f0e11348 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:08.422 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': '3D3C49D5-76D8-4BE2-9DC6-46D5C26A928D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16039), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:08.554 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:08.554 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:08.554 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:08.565 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:08.565 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3D3C49D5-76D8-4BE2-9DC6-46D5C26A928D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110496), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:08.565 4076 3051113164120 MainThread DEBUG os_win._utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.521942377090454 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:08.582 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-ea57ea2e-191b-461a-a0fb-2d58ad9e255f - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:08.582 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-ea57ea2e-191b-461a-a0fb-2d58ad9e255f - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:08.582 4076 3051113160776 GreenThread-42 DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Updating instance_info_cache with network_info: [{"id": "66ec8cb2-5a45-471f-a9e4-5e85bfaab509", "address": "fa:16:3e:08:d3:96", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap66ec8cb2-5a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:08.608 4076 3051113160776 GreenThread-42 DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Releasing lock "refresh_cache-082d9c5d-01d4-43af-8493-80ba655501fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:08.608 4076 3051113160776 GreenThread-42 DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Instance network_info: |[{"id": "66ec8cb2-5a45-471f-a9e4-5e85bfaab509", "address": "fa:16:3e:08:d3:96", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap66ec8cb2-5a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:08.608 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Preparing to wait for external event network-vif-plugged-66ec8cb2-5a45-471f-a9e4-5e85bfaab509 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:08.622 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:08.622 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:08.622 4076 3051113160952 MainThread DEBUG os_win.utils.compute.vmutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating VM instance-00000037 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:08.622 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-082d9c5d-01d4-43af-8493-80ba655501fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:08.622 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Refreshing network info cache for port 66ec8cb2-5a45-471f-a9e4-5e85bfaab509 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:08.638 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-ea57ea2e-191b-461a-a0fb-2d58ad9e255f - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:08.650 4076 3051113163768 MainThread INFO nova.compute.manager [req-ea57ea2e-191b-461a-a0fb-2d58ad9e255f - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:08.860 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Updated VIF entry in instance network info cache for port 1e195453-3f7d-47ae-b9de-c2ba027332be. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:08.860 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Updating instance_info_cache with network_info: [{"id": "1e195453-3f7d-47ae-b9de-c2ba027332be", "address": "fa:16:3e:84:ef:aa", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1e195453-3f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:08.882 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-04ac9dc0-83ba-408c-8949-6d0a0ac01c06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4bdc3835-ba14-4e1b-bd49-23ec32f15291" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:09.068 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Successfully created port: cc3bb767-8404-4c26-903f-4b725fa73128 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:09.520 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:09.587 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': '607985CF-F0F4-4C62-B070-DBE1C80CE68C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16503), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:09.733 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:09.733 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:09.744 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:09.744 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:09.744 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '607985CF-F0F4-4C62-B070-DBE1C80CE68C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115067), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:09.744 4076 3051113164120 MainThread DEBUG os_win._utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.701612949371338 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:09.764 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-3e6e190b-727b-4be5-ab94-b5550a03cdca - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:09.764 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-3e6e190b-727b-4be5-ab94-b5550a03cdca - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:09.805 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-3e6e190b-727b-4be5-ab94-b5550a03cdca - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:09.828 4076 3051113162008 MainThread INFO nova.compute.manager [req-3e6e190b-727b-4be5-ab94-b5550a03cdca - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:10.667 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Successfully updated port: 7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:10.723 4076 3051113162360 GreenThread-43 DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Acquired lock "refresh_cache-c7fdc9fc-d133-4f12-ab12-ff417c249240" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:10.723 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:10.790 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': '0703F7A8-C00B-4743-8882-69766DF98B08', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17076), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:10.917 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:10.917 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:10.917 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:10.917 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:10.930 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0703F7A8-C00B-4743-8882-69766DF98B08', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111066), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:10.930 4076 3051113164120 MainThread DEBUG os_win._utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.887051343917847 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:10.947 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-4d192d10-bbaa-4f43-947e-dd9d0d70a579 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:10.947 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-4d192d10-bbaa-4f43-947e-dd9d0d70a579 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:11.004 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-4d192d10-bbaa-4f43-947e-dd9d0d70a579 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:11.040 4076 3051113162184 MainThread INFO nova.compute.manager [req-4d192d10-bbaa-4f43-947e-dd9d0d70a579 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:11.163 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:11.274 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Updated VIF entry in instance network info cache for port 66ec8cb2-5a45-471f-a9e4-5e85bfaab509. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:11.274 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Updating instance_info_cache with network_info: [{"id": "66ec8cb2-5a45-471f-a9e4-5e85bfaab509", "address": "fa:16:3e:08:d3:96", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap66ec8cb2-5a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:11.320 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-32b04ea0-fa8f-4bd4-bac6-70945a68062b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-082d9c5d-01d4-43af-8493-80ba655501fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:11.867 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Successfully updated port: cc3bb767-8404-4c26-903f-4b725fa73128 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:11.904 4076 3051049643352 GreenThread-44 DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Acquired lock "refresh_cache-832a0158-f115-480e-a24e-78506e51b9f6" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:11.904 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:11.945 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': '16425151-5549-48C4-A163-938A936F4D03', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14236), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:12.086 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000035\' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000035\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)instance-000000351E3F49F0-DB08-4A17-903C-76B927D8AE6E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '16425151-5549-48C4-A163-938A936F4D03', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110752), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\nNot enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)\n\n'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E)", 'ErrorSummaryDescription': "'instance-00000035' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000035.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:12.086 4076 3051113164120 MainThread ERROR nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Failed to change vm state of instance-00000035 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. 2021-09-10 16:58:12.086 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Got request to destroy instance 2021-09-10 16:58:12.086 4076 3051113161480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:12.086 4076 3051113161480 MainThread INFO nova.compute.manager [-] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] VM Stopped (Lifecycle Event) 2021-09-10 16:58:12.086 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:12.116 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:12.132 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-c0f94686-ccb2-4b98-84ae-f8b36c6d7228 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:12.132 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-c0f94686-ccb2-4b98-84ae-f8b36c6d7228 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:12.181 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-c0f94686-ccb2-4b98-84ae-f8b36c6d7228 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:12.212 4076 3051113161480 MainThread INFO nova.compute.manager [req-c0f94686-ccb2-4b98-84ae-f8b36c6d7228 - - - - -] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:12.247 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:12.258 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Received event network-changed-cc3bb767-8404-4c26-903f-4b725fa73128 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:12.258 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Refreshing instance network info cache due to event network-changed-cc3bb767-8404-4c26-903f-4b725fa73128. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:12.930 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Received event network-changed-7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:12.930 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Refreshing instance network info cache due to event network-changed-7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:13.259 4076 3051049643352 GreenThread-44 DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Updating instance_info_cache with network_info: [{"id": "cc3bb767-8404-4c26-903f-4b725fa73128", "address": "fa:16:3e:b8:9b:68", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcc3bb767-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:13.276 4076 3051049643352 GreenThread-44 DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Releasing lock "refresh_cache-832a0158-f115-480e-a24e-78506e51b9f6" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:13.276 4076 3051049643352 GreenThread-44 DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Instance network_info: |[{"id": "cc3bb767-8404-4c26-903f-4b725fa73128", "address": "fa:16:3e:b8:9b:68", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcc3bb767-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:13.276 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-832a0158-f115-480e-a24e-78506e51b9f6" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:13.276 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Refreshing network info cache for port cc3bb767-8404-4c26-903f-4b725fa73128 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:13.301 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Preparing to wait for external event network-vif-plugged-cc3bb767-8404-4c26-903f-4b725fa73128 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:13.301 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:13.301 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:13.301 4076 3051113161656 MainThread DEBUG os_win.utils.compute.vmutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating VM instance-00000039 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:13.737 4076 3051113162360 GreenThread-43 DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Updating instance_info_cache with network_info: [{"id": "7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7", "address": "fa:16:3e:d0:0a:ca", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f3ea6cc-0d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:13.763 4076 3051113162360 GreenThread-43 DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Releasing lock "refresh_cache-c7fdc9fc-d133-4f12-ab12-ff417c249240" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:13.763 4076 3051113162360 GreenThread-43 DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Instance network_info: |[{"id": "7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7", "address": "fa:16:3e:d0:0a:ca", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f3ea6cc-0d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:13.763 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-c7fdc9fc-d133-4f12-ab12-ff417c249240" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:13.763 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Refreshing network info cache for port 7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:13.763 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Preparing to wait for external event network-vif-plugged-7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:13.763 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:13.763 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:13.763 4076 3051113163064 MainThread DEBUG os_win.utils.compute.vmutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Creating VM instance-00000038 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:13.995 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:13.995 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:14.008 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:14.026 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Successfully changed vm state of instance-00000035 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:14.026 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Successfully changed state of VM instance-00000035 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:14.076 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Waiting for WMI job: {'InstanceID': '80D5309F-B6C1-40B0-8628-6FE7E6712E90', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15389), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:14.180 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] WMI job succeeded: {'InstanceID': '80D5309F-B6C1-40B0-8628-6FE7E6712E90', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29835), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:14.180 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:14.180 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Removing directory: C:\openstack\instances\instance-00000035 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:14.199 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:14.199 4076 3051113164120 MainThread ERROR nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Traceback (most recent call last): 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(*args, **kwargs) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] admin_password, network_info, block_device_info) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(self, *args, **kwds) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] should_plug_vifs=False) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'req_state': req_state}) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] time.sleep(sleep_time) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return f(*args, **kwargs) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return self._wait_for_job(job_path) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] error_desc=err_desc) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During handling of the above exception, another exception occurred: 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Traceback (most recent call last): 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] yield resources 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] accel_info=accel_info) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(*args, **kwargs) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] admin_password, network_info, block_device_info) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(self, *args, **kwds) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] should_plug_vifs=False) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'req_state': req_state}) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] time.sleep(sleep_time) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return f(*args, **kwargs) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return self._wait_for_job(job_path) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] error_desc=err_desc) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.199 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.199 4076 3051113164120 MainThread INFO nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Terminating instance 2021-09-10 16:58:14.199 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:14.199 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Got request to destroy instance 2021-09-10 16:58:14.212 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:14.212 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:14.212 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "instance-00000035" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:14.212 4076 3051113164120 MainThread INFO nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:14.212 4076 3051113164120 MainThread DEBUG nova.compute.claims [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:14.212 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:14.273 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:14.401 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:14.433 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:14.640 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:14.677 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:14.692 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:14.692 4076 3051113164120 MainThread ERROR nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Traceback (most recent call last): 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(*args, **kwargs) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] admin_password, network_info, block_device_info) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(self, *args, **kwds) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] should_plug_vifs=False) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'req_state': req_state}) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] time.sleep(sleep_time) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return f(*args, **kwargs) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return self._wait_for_job(job_path) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] error_desc=err_desc) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] During handling of the above exception, another exception occurred: 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Traceback (most recent call last): 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] accel_info=accel_info) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(*args, **kwargs) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] admin_password, network_info, block_device_info) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return function(self, *args, **kwds) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] should_plug_vifs=False) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'req_state': req_state}) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] time.sleep(sleep_time) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self.force_reraise() 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] raise self.value 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return f(*args, **kwargs) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] return self._wait_for_job(job_path) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] error_desc=err_desc) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. 2021-09-10 16:58:14.692 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] 2021-09-10 16:58:14.692 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:14.692 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Build of instance 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000035' failed to start. Not enough memory in the system to start the virtual machine instance-00000035. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000035' failed to start. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Not enough memory in the system to start the virtual machine instance-00000035 with ram size 1024 megabytes. (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) 'instance-00000035' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1E3F49F0-DB08-4A17-903C-76B927D8AE6E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:14.692 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:14.705 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:14.705 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:14.705 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:14.766 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:15.180 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-4c4d8aa9-6902-4a94-a46d-de874c67b3d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Received event network-vif-plugged-1e195453-3f7d-47ae-b9de-c2ba027332be external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:15.180 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-4c4d8aa9-6902-4a94-a46d-de874c67b3d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:15.180 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-4c4d8aa9-6902-4a94-a46d-de874c67b3d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:15.180 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-4c4d8aa9-6902-4a94-a46d-de874c67b3d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Processing event network-vif-plugged-1e195453-3f7d-47ae-b9de-c2ba027332be _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:15.215 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Using config drive for instance 2021-09-10 16:58:15.441 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Updated VIF entry in instance network info cache for port cc3bb767-8404-4c26-903f-4b725fa73128. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:15.441 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Updating instance_info_cache with network_info: [{"id": "cc3bb767-8404-4c26-903f-4b725fa73128", "address": "fa:16:3e:b8:9b:68", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapcc3bb767-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:15.461 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-dffe2a79-2149-45ba-9dd6-c5dcc91192d9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-832a0158-f115-480e-a24e-78506e51b9f6" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:16.258 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Creating config drive at C:\openstack\instances\instance-00000036\configdrive.iso 2021-09-10 16:58:16.307 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000036\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpai1ngjv8 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:16.594 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Updated VIF entry in instance network info cache for port 7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:16.594 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Updating instance_info_cache with network_info: [{"id": "7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7", "address": "fa:16:3e:d0:0a:ca", "network": {"id": "99282f2c-b612-4703-8180-12ed8aae404f", "bridge": null, "label": "tempest-ListServerFiltersTestJSON-836490928-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cc8daf6ce3114d6aa694f16669c8ed20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f3ea6cc-0d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:16.617 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-809851a7-e0fc-4558-97af-5f09469ba876 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-c7fdc9fc-d133-4f12-ab12-ff417c249240" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:16.630 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000036\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpai1ngjv8" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:16.882 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:16.908 4076 3051113164120 MainThread INFO nova.compute.manager [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] [instance: 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538] Took 2.20 seconds to deallocate network for instance. 2021-09-10 16:58:17.059 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Setting boot order for instance: instance-00000036: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:17.106 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:17.151 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '4C1B68D5-3E7C-45F7-981F-E85005967FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15553), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:17.260 4076 3051113164120 MainThread INFO nova.scheduler.client.report [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Deleted allocations for instance 0713ad83-2fd1-4b7f-bbe2-e9035bfc3538 2021-09-10 16:58:17.289 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:17.289 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:17.289 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:17.289 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:17.289 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-61901d14-7be4-43ae-984e-c7a3d360bfc4 5caa9c73de9d44b4a4001f521e76cb24 6153c0b18ada42b8b1fc8a04fbd01d24 - default default] Lock "0713ad83-2fd1-4b7f-bbe2-e9035bfc3538" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.281s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:17.289 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4C1B68D5-3E7C-45F7-981F-E85005967FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112534), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:17.289 4076 3051113164648 MainThread DEBUG os_win._utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18306565284729004 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:17.383 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1ab893b8-8f56-4151-bae3-1751bb08deea - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:17.383 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1ab893b8-8f56-4151-bae3-1751bb08deea - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:17.426 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1ab893b8-8f56-4151-bae3-1751bb08deea - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:17.460 4076 3051049643352 MainThread INFO nova.compute.manager [req-1ab893b8-8f56-4151-bae3-1751bb08deea - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:18.335 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '99C71167-FB61-4393-99A5-F0764F5590D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14958), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:18.463 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:18.463 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:18.463 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:18.463 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:18.492 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-62c51488-7a8d-47b7-a788-12883d5379a4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:18.492 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-62c51488-7a8d-47b7-a788-12883d5379a4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:18.494 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '99C71167-FB61-4393-99A5-F0764F5590D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111180), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:18.494 4076 3051113164648 MainThread DEBUG os_win._utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3887097835540771 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:18.529 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-62c51488-7a8d-47b7-a788-12883d5379a4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:18.571 4076 3051049643352 MainThread INFO nova.compute.manager [req-62c51488-7a8d-47b7-a788-12883d5379a4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:19.519 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '35F2F924-AE38-41A4-896D-2C41407F850B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15512), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:19.662 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:19.662 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:19.662 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:19.662 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:19.668 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '35F2F924-AE38-41A4-896D-2C41407F850B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111442), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:19.668 4076 3051113164648 MainThread DEBUG os_win._utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5621166229248047 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:19.695 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-fd8d29eb-0d08-42a1-ba78-c6b6ec628983 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:19.695 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-fd8d29eb-0d08-42a1-ba78-c6b6ec628983 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:19.728 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-fd8d29eb-0d08-42a1-ba78-c6b6ec628983 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:19.758 4076 3051049643352 MainThread INFO nova.compute.manager [req-fd8d29eb-0d08-42a1-ba78-c6b6ec628983 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:20.417 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:20.448 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:58:20.648 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:20.648 4076 3051113161304 MainThread DEBUG nova.virt.hardware [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:58:20.648 4076 3051113161304 MainThread INFO nova.compute.claims [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Claim successful on node n-h1-808474-1 2021-09-10 16:58:20.688 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '0C9A0BEB-7BF0-4DF1-AE82-7CA26B883A8F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15336), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:20.822 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:20.822 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:20.822 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:20.839 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:20.839 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0C9A0BEB-7BF0-4DF1-AE82-7CA26B883A8F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110714), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:20.839 4076 3051113164648 MainThread DEBUG os_win._utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7331438064575195 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:20.853 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-e096242f-8095-4008-9949-05bef8d4fca4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:20.853 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-e096242f-8095-4008-9949-05bef8d4fca4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:20.867 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.hostops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:20.867 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.hostops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:20.903 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-e096242f-8095-4008-9949-05bef8d4fca4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:20.930 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-438c63d0-6415-4da9-a713-8605d5d9c4e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Received event network-vif-plugged-66ec8cb2-5a45-471f-a9e4-5e85bfaab509 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:20.930 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-438c63d0-6415-4da9-a713-8605d5d9c4e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:20.930 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-438c63d0-6415-4da9-a713-8605d5d9c4e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:20.930 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-438c63d0-6415-4da9-a713-8605d5d9c4e4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Processing event network-vif-plugged-66ec8cb2-5a45-471f-a9e4-5e85bfaab509 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:20.930 4076 3051049643352 MainThread INFO nova.compute.manager [req-e096242f-8095-4008-9949-05bef8d4fca4 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:20.976 4076 3051113160952 MainThread INFO nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Using config drive for instance 2021-09-10 16:58:21.098 4076 3051113161304 MainThread DEBUG nova.compute.provider_tree [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:21.106 4076 3051113161304 MainThread DEBUG nova.scheduler.client.report [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:21.106 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:21.106 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:21.203 4076 3051113161304 MainThread DEBUG nova.compute.utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:21.220 4076 3051113162360 GreenThread-45 DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:21.220 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:21.234 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:21.429 4076 3051113160952 MainThread INFO nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Creating config drive at C:\openstack\instances\instance-00000037\configdrive.iso 2021-09-10 16:58:21.460 4076 3051113160952 MainThread DEBUG oslo_concurrency.processutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000037\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkzr2_0ge execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:21.588 4076 3051113162360 GreenThread-45 DEBUG nova.policy [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a1711a7b02c2433dacfb4017cd0a812c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b9dfa46b614e449b6aff88db2cfd6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:21.607 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:21.607 4076 3051113161304 MainThread INFO nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Spawning new instance 2021-09-10 16:58:21.636 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG nova.block_device [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG os_win.utils.pathutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating directory: C:\openstack\instances\instance-0000003a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:58:21.636 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000003a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:58:21.664 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:58:21.681 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Resizing VHD C:\openstack\instances\instance-0000003a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:58:21.767 4076 3051113160952 MainThread DEBUG oslo_concurrency.processutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000037\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkzr2_0ge" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:21.824 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:21.824 4076 3051113161480 MainThread DEBUG nova.virt.hardware [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:58:21.824 4076 3051113161480 MainThread INFO nova.compute.claims [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Claim successful on node n-h1-808474-1 2021-09-10 16:58:21.874 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'F78D4731-EA18-44A6-9091-A4043BDB27A2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15395), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:22.010 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:22.010 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:22.010 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:22.010 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:22.010 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F78D4731-EA18-44A6-9091-A4043BDB27A2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114238), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:22.010 4076 3051113164648 MainThread DEBUG os_win._utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.904249668121338 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:22.010 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.hostops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:22.031 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.hostops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:22.048 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-0a1b7739-0e2d-45e1-b0b7-9e2e18e95185 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:22.048 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-0a1b7739-0e2d-45e1-b0b7-9e2e18e95185 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:22.102 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-0a1b7739-0e2d-45e1-b0b7-9e2e18e95185 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:22.163 4076 3051049643352 MainThread INFO nova.compute.manager [req-0a1b7739-0e2d-45e1-b0b7-9e2e18e95185 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:22.209 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Setting boot order for instance: instance-00000037: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:22.244 4076 3051113161480 MainThread DEBUG nova.compute.provider_tree [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:22.275 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:22.275 4076 3051113161480 MainThread DEBUG nova.scheduler.client.report [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:22.275 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:22.275 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:22.306 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '2D1323BA-E973-4FF5-867C-00A83AFB72B2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14568), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:22.445 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:22.445 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:22.445 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:22.445 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:22.445 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2D1323BA-E973-4FF5-867C-00A83AFB72B2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114246), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:22.445 4076 3051113160952 MainThread DEBUG os_win._utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17040014266967773 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:22.476 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1dc31e07-011d-47d2-b222-42231b1a4842 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:22.476 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-1dc31e07-011d-47d2-b222-42231b1a4842 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:22.500 4076 3051113161480 MainThread DEBUG nova.compute.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:22.500 4076 3051113162184 GreenThread-46 DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:22.500 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:22.532 4076 3051113161480 MainThread DEBUG nova.compute.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:22.539 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-1dc31e07-011d-47d2-b222-42231b1a4842 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:22.566 4076 3051113161480 MainThread DEBUG nova.compute.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:22.590 4076 3051049643352 MainThread INFO nova.compute.manager [req-1dc31e07-011d-47d2-b222-42231b1a4842 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:22.602 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:22.703 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:22.712 4076 3051113161480 MainThread INFO nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Booting with volume 29c54fbe-e8c1-401b-a427-b0b007cae3a7 at /dev/sda 2021-09-10 16:58:22.726 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:58:22.884 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:58:22.884 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:58:22.884 4076 3051113161480 MainThread DEBUG nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating existing volume attachment record: 797edfeb-1838-4fdf-9583-9cfe154f3f38 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:58:22.910 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:22.910 4076 3051113163768 MainThread DEBUG nova.virt.hardware [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:58:22.910 4076 3051113163768 MainThread INFO nova.compute.claims [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Claim successful on node n-h1-808474-1 2021-09-10 16:58:23.039 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'C1054502-8C01-4ADC-BF61-7CE90ACA92E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16246), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:23.102 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:23.102 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:23.174 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:23.174 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] VM Stopped (Lifecycle Event) 2021-09-10 16:58:23.174 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:23.174 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.174 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000036\' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000036\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)instance-000000364E739DE0-638B-45C0-8112-35591E66D34D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C1054502-8C01-4ADC-BF61-7CE90ACA92E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113457), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\nNot enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)\n\n'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D)", 'ErrorSummaryDescription': "'instance-00000036' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000036.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:23.174 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Failed to change vm state of instance-00000036 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. 2021-09-10 16:58:23.174 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Got request to destroy instance 2021-09-10 16:58:23.194 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-5f20bfc4-bb66-4ab4-bd2f-c6b8aff2d9a6 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:23.194 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-5f20bfc4-bb66-4ab4-bd2f-c6b8aff2d9a6 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:23.242 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-5f20bfc4-bb66-4ab4-bd2f-c6b8aff2d9a6 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:23.262 4076 3051113160776 MainThread INFO nova.compute.manager [req-5f20bfc4-bb66-4ab4-bd2f-c6b8aff2d9a6 - - - - -] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:23.304 4076 3051113162184 GreenThread-46 DEBUG nova.policy [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e257d97e390840179aaf42ec0308f2de', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f00087122c344aa8a15842d2b03f9a16', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:23.338 4076 3051113163768 MainThread DEBUG nova.compute.provider_tree [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:23.363 4076 3051113163768 MainThread DEBUG nova.scheduler.client.report [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:23.382 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.382 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:58:23.466 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '8C809341-5D3B-4C15-8587-EC399165F26E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15291), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:23.605 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:23.605 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.605 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:23.605 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:23.623 4076 3051113163768 MainThread DEBUG nova.compute.utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:58:23.623 4076 3051113162536 GreenThread-47 DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:58:23.623 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:58:23.637 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-cddb2b30-943e-4bae-b0ea-a4f90db40be4 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:23.637 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-cddb2b30-943e-4bae-b0ea-a4f90db40be4 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:23.651 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:58:23.697 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-cddb2b30-943e-4bae-b0ea-a4f90db40be4 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:23.726 4076 3051113160776 MainThread INFO nova.compute.manager [req-cddb2b30-943e-4bae-b0ea-a4f90db40be4 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:23.726 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8C809341-5D3B-4C15-8587-EC399165F26E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115022), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:23.726 4076 3051113160952 MainThread DEBUG os_win._utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4517405033111572 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:23.924 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-4cb578d1-2130-4e17-b129-8764d4d935eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Received event network-vif-plugged-cc3bb767-8404-4c26-903f-4b725fa73128 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:23.924 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-4cb578d1-2130-4e17-b129-8764d4d935eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:23.924 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-4cb578d1-2130-4e17-b129-8764d4d935eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.924 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-4cb578d1-2130-4e17-b129-8764d4d935eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Processing event network-vif-plugged-cc3bb767-8404-4c26-903f-4b725fa73128 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:23.965 4076 3051113161656 MainThread INFO nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Using config drive for instance 2021-09-10 16:58:23.965 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:23.965 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Spawning new instance 2021-09-10 16:58:23.991 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:23.991 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.991 4076 3051113163768 MainThread DEBUG nova.block_device [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:23.997 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:23.997 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:23.997 4076 3051113163768 MainThread DEBUG os_win.utils.pathutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Creating directory: C:\openstack\instances\instance-0000003c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:58:23.997 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000003c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:58:24.025 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Resizing VHD C:\openstack\instances\instance-0000003c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:58:24.197 4076 3051113162536 GreenThread-47 DEBUG nova.policy [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd62454119feb46319b4a349e02ddd6df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a13940e712fe464c8a1b2c3732982edb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:58:24.342 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Successfully created port: 2ab80196-8434-4424-b44a-f47ed0cf89ad _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:24.758 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'AD333FF5-D029-4D16-987C-C95C8E5F311C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15413), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:24.893 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:24.893 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:24.893 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:24.893 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:24.903 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AD333FF5-D029-4D16-987C-C95C8E5F311C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114946), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:24.903 4076 3051113160952 MainThread DEBUG os_win._utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6282618045806885 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:24.931 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-5543577b-9b01-4eef-9d06-73cdf3d6fd86 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:24.931 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-5543577b-9b01-4eef-9d06-73cdf3d6fd86 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:24.983 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-5543577b-9b01-4eef-9d06-73cdf3d6fd86 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:24.995 4076 3051055216776 MainThread INFO nova.compute.manager [req-5543577b-9b01-4eef-9d06-73cdf3d6fd86 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:25.102 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:25.102 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:25.102 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:25.102 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed vm state of instance-00000036 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:25.102 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed state of VM instance-00000036 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:25.164 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'C99BE28A-8B4D-437E-B907-6844C30D0DAB', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14476), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:25.288 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job succeeded: {'InstanceID': 'C99BE28A-8B4D-437E-B907-6844C30D0DAB', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30330), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:25.288 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:25.288 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Removing directory: C:\openstack\instances\instance-00000036 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:25.288 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:25.288 4076 3051113164648 MainThread ERROR nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Traceback (most recent call last): 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(*args, **kwargs) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] admin_password, network_info, block_device_info) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(self, *args, **kwds) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] should_plug_vifs=False) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'req_state': req_state}) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] time.sleep(sleep_time) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return f(*args, **kwargs) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return self._wait_for_job(job_path) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] error_desc=err_desc) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During handling of the above exception, another exception occurred: 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Traceback (most recent call last): 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] yield resources 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] accel_info=accel_info) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(*args, **kwargs) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] admin_password, network_info, block_device_info) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(self, *args, **kwds) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] should_plug_vifs=False) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'req_state': req_state}) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] time.sleep(sleep_time) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return f(*args, **kwargs) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return self._wait_for_job(job_path) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] error_desc=err_desc) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.288 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.288 4076 3051113164648 MainThread INFO nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Terminating instance 2021-09-10 16:58:25.288 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:25.288 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Got request to destroy instance 2021-09-10 16:58:25.319 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:25.319 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:25.319 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000036" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:25.319 4076 3051113164648 MainThread INFO nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:25.329 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:25.329 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:25.522 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:25.535 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:25.569 4076 3051113161656 MainThread INFO nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Creating config drive at C:\openstack\instances\instance-00000039\configdrive.iso 2021-09-10 16:58:25.612 4076 3051113161656 MainThread DEBUG oslo_concurrency.processutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000039\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpp4p1ch42 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:25.809 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Successfully updated port: 2ab80196-8434-4424-b44a-f47ed0cf89ad _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:25.829 4076 3051113162360 GreenThread-45 DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Acquired lock "refresh_cache-1f6d373b-5624-411c-9545-511d98e94f86" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:25.829 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:25.868 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully created port: 4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:25.914 4076 3051113161656 MainThread DEBUG oslo_concurrency.processutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000039\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpp4p1ch42" returned: 0 in 0.296s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:25.934 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'C76C6D7B-6BAC-454F-A7F3-611F0F616CA1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15727), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:25.957 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.641s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:25.977 4076 3051113164648 MainThread ERROR nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Traceback (most recent call last): 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(*args, **kwargs) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] admin_password, network_info, block_device_info) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(self, *args, **kwds) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] should_plug_vifs=False) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'req_state': req_state}) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] time.sleep(sleep_time) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return f(*args, **kwargs) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return self._wait_for_job(job_path) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] error_desc=err_desc) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] During handling of the above exception, another exception occurred: 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Traceback (most recent call last): 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] accel_info=accel_info) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(*args, **kwargs) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] admin_password, network_info, block_device_info) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return function(self, *args, **kwds) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] should_plug_vifs=False) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'req_state': req_state}) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] time.sleep(sleep_time) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self.force_reraise() 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] raise self.value 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return f(*args, **kwargs) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] return self._wait_for_job(job_path) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] error_desc=err_desc) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. 2021-09-10 16:58:25.977 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Build of instance 4bdc3835-ba14-4e1b-bd49-23ec32f15291 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000036' failed to start. Not enough memory in the system to start the virtual machine instance-00000036. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000036' failed to start. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Not enough memory in the system to start the virtual machine instance-00000036 with ram size 1024 megabytes. (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) 'instance-00000036' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E739DE0-638B-45C0-8112-35591E66D34D) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:25.977 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:25.977 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Successfully created port: 20f19fa9-4c3e-43e8-8e22-6987b697fd3b _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:26.008 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:26.056 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:26.056 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:26.056 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:26.056 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:26.070 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C76C6D7B-6BAC-454F-A7F3-611F0F616CA1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115591), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:26.070 4076 3051113160952 MainThread DEBUG os_win._utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7953782081604004 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:26.088 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-7b6c42bf-5f1d-4b44-bc16-65600b425f5c - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:26.088 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-7b6c42bf-5f1d-4b44-bc16-65600b425f5c - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:26.151 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-7b6c42bf-5f1d-4b44-bc16-65600b425f5c - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:26.177 4076 3051055217128 MainThread INFO nova.compute.manager [req-7b6c42bf-5f1d-4b44-bc16-65600b425f5c - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:26.340 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Setting boot order for instance: instance-00000039: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:26.389 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:26.429 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'B934602D-6E12-4AC1-969E-C5B25C26BCFE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15134), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:26.566 4076 3051113161832 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:26.566 4076 3051113161832 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:26.571 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:26.571 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:26.571 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B934602D-6E12-4AC1-969E-C5B25C26BCFE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115688), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:26.571 4076 3051113161656 MainThread DEBUG os_win._utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18202424049377441 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:26.588 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-2a109b47-d4bc-4d0c-a65c-67a4b8989ad6 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:26.588 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-2a109b47-d4bc-4d0c-a65c-67a4b8989ad6 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:26.628 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-2a109b47-d4bc-4d0c-a65c-67a4b8989ad6 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:26.663 4076 3051113161832 MainThread INFO nova.compute.manager [req-2a109b47-d4bc-4d0c-a65c-67a4b8989ad6 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:27.009 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Received event network-changed-2ab80196-8434-4424-b44a-f47ed0cf89ad external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:27.009 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Refreshing instance network info cache due to event network-changed-2ab80196-8434-4424-b44a-f47ed0cf89ad. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:27.084 4076 3051113162360 GreenThread-45 DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Updating instance_info_cache with network_info: [{"id": "2ab80196-8434-4424-b44a-f47ed0cf89ad", "address": "fa:16:3e:9f:be:59", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2ab80196-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:27.084 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-f1d87118-a58f-4a54-8cae-dfbde704fa8d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Received event network-vif-plugged-7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:27.084 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-f1d87118-a58f-4a54-8cae-dfbde704fa8d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:27.084 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-f1d87118-a58f-4a54-8cae-dfbde704fa8d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:27.084 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-f1d87118-a58f-4a54-8cae-dfbde704fa8d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Processing event network-vif-plugged-7f3ea6cc-0d3a-4d84-bfc5-88187dda64f7 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:27.102 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '5CF8FC55-1EAA-40AF-ABC1-DDEDA778160D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=31397), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:27.132 4076 3051113162360 GreenThread-45 DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Releasing lock "refresh_cache-1f6d373b-5624-411c-9545-511d98e94f86" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:27.132 4076 3051113162360 GreenThread-45 DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Instance network_info: |[{"id": "2ab80196-8434-4424-b44a-f47ed0cf89ad", "address": "fa:16:3e:9f:be:59", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2ab80196-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:27.132 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-1f6d373b-5624-411c-9545-511d98e94f86" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:27.132 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Refreshing network info cache for port 2ab80196-8434-4424-b44a-f47ed0cf89ad _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:27.132 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Preparing to wait for external event network-vif-plugged-2ab80196-8434-4424-b44a-f47ed0cf89ad prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:27.132 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:27.132 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:27.150 4076 3051113161304 MainThread DEBUG os_win.utils.compute.vmutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating VM instance-0000003a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:27.150 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Using config drive for instance 2021-09-10 16:58:27.232 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:27.232 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:27.232 4076 3051093170488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:27.232 4076 3051093170488 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:27.232 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5CF8FC55-1EAA-40AF-ABC1-DDEDA778160D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115328), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:27.232 4076 3051113160952 MainThread DEBUG os_win._utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.957250595092773 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:27.273 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-7030b123-e1f5-44cd-9329-1c16d601a900 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:27.273 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-7030b123-e1f5-44cd-9329-1c16d601a900 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:27.311 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-7030b123-e1f5-44cd-9329-1c16d601a900 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:27.336 4076 3051093170488 MainThread INFO nova.compute.manager [req-7030b123-e1f5-44cd-9329-1c16d601a900 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:27.470 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Successfully updated port: 20f19fa9-4c3e-43e8-8e22-6987b697fd3b _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:27.493 4076 3051113162536 GreenThread-47 DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Acquired lock "refresh_cache-04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:27.493 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:58:27.593 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Creating config drive at C:\openstack\instances\instance-00000038\configdrive.iso 2021-09-10 16:58:27.619 4076 3051113163064 MainThread DEBUG oslo_concurrency.processutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000038\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmhdn46rb execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:27.774 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:27.774 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'FB530E04-E526-4C21-AD29-CACA78D1F8F8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19781), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:27.774 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:27.804 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:27.804 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:27.804 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:27.804 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:27.814 4076 3051113164648 MainThread INFO nova.compute.manager [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4bdc3835-ba14-4e1b-bd49-23ec32f15291] Took 1.84 seconds to deallocate network for instance. 2021-09-10 16:58:27.823 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-881f0be8-da8c-4c3a-b504-3615c6ffc2a4 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:27.836 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-881f0be8-da8c-4c3a-b504-3615c6ffc2a4 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:27.871 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-881f0be8-da8c-4c3a-b504-3615c6ffc2a4 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:27.917 4076 3051055217128 MainThread INFO nova.compute.manager [req-881f0be8-da8c-4c3a-b504-3615c6ffc2a4 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:27.932 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FB530E04-E526-4C21-AD29-CACA78D1F8F8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114727), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:27.932 4076 3051113161656 MainThread DEBUG os_win._utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.5423030853271484 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:27.932 4076 3051113163064 MainThread DEBUG oslo_concurrency.processutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000038\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmhdn46rb" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:28.056 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Deleted allocations for instance 4bdc3835-ba14-4e1b-bd49-23ec32f15291 2021-09-10 16:58:28.132 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-175715cb-af84-422d-af9d-83190916c6c4 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4bdc3835-ba14-4e1b-bd49-23ec32f15291" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:28.293 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '619C72A4-B7B3-40CE-8B4C-8383906A280E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15135), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:28.321 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully created port: 1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:28.337 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:28.387 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Setting boot order for instance: instance-00000038: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:28.436 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:28.436 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:28.436 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:28.436 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] VM Stopped (Lifecycle Event) 2021-09-10 16:58:28.445 4076 3051113160952 MainThread ERROR os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000037\' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000037\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)instance-00000037CD065D42-35B1-4A7A-947D-866C4098026C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '619C72A4-B7B3-40CE-8B4C-8383906A280E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118105), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\nNot enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)\n\n'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C)", 'ErrorSummaryDescription': "'instance-00000037' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000037.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:28.445 4076 3051113160952 MainThread ERROR nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Failed to change vm state of instance-00000037 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. 2021-09-10 16:58:28.445 4076 3051113160952 MainThread INFO nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Got request to destroy instance 2021-09-10 16:58:28.445 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:28.474 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-61f1547b-0902-44a4-95dc-7ba0fdea8316 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:28.474 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-61f1547b-0902-44a4-95dc-7ba0fdea8316 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:28.484 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'A441A448-8D1A-4980-8EE7-6511DF7E0ABC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15099), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:28.527 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-61f1547b-0902-44a4-95dc-7ba0fdea8316 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:28.577 4076 3051113164648 MainThread INFO nova.compute.manager [req-61f1547b-0902-44a4-95dc-7ba0fdea8316 - - - - -] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:28.633 4076 3051113162536 GreenThread-47 DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Updating instance_info_cache with network_info: [{"id": "20f19fa9-4c3e-43e8-8e22-6987b697fd3b", "address": "fa:16:3e:b4:44:ec", "network": {"id": "6b3d4f05-5619-41ec-8def-1bd17c9bcd14", "bridge": null, "label": "tempest-ServerRescueTestJSON-1067413489-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a13940e712fe464c8a1b2c3732982edb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap20f19fa9-4c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:28.633 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:28.633 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:28.633 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:28.633 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:28.633 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A441A448-8D1A-4980-8EE7-6511DF7E0ABC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120669), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:28.633 4076 3051113163064 MainThread DEBUG os_win._utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18865656852722168 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:28.654 4076 3051113162536 GreenThread-47 DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Releasing lock "refresh_cache-04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:28.654 4076 3051113162536 GreenThread-47 DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Instance network_info: |[{"id": "20f19fa9-4c3e-43e8-8e22-6987b697fd3b", "address": "fa:16:3e:b4:44:ec", "network": {"id": "6b3d4f05-5619-41ec-8def-1bd17c9bcd14", "bridge": null, "label": "tempest-ServerRescueTestJSON-1067413489-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a13940e712fe464c8a1b2c3732982edb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap20f19fa9-4c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:58:28.654 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Preparing to wait for external event network-vif-plugged-20f19fa9-4c3e-43e8-8e22-6987b697fd3b prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:58:28.654 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:28.654 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:28.654 4076 3051113163768 MainThread DEBUG os_win.utils.compute.vmutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Creating VM instance-0000003c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:58:28.737 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-64fc93c4-2b94-4d4c-9cc8-905d61a8babb - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:28.737 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-64fc93c4-2b94-4d4c-9cc8-905d61a8babb - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:28.779 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-64fc93c4-2b94-4d4c-9cc8-905d61a8babb - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:28.823 4076 3051113160776 MainThread INFO nova.compute.manager [req-64fc93c4-2b94-4d4c-9cc8-905d61a8babb - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:28.838 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Updated VIF entry in instance network info cache for port 2ab80196-8434-4424-b44a-f47ed0cf89ad. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:28.838 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Updating instance_info_cache with network_info: [{"id": "2ab80196-8434-4424-b44a-f47ed0cf89ad", "address": "fa:16:3e:9f:be:59", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2ab80196-84", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:28.867 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-1360fa29-9a40-47a6-a49e-e1accbb64b00 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-1f6d373b-5624-411c-9545-511d98e94f86" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:28.971 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '1ECEB40C-F5F6-4FA9-9729-4D7F3738B1EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14595), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:29.103 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:29.103 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:29.103 4076 3051113161832 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:29.103 4076 3051113161832 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:29.103 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1ECEB40C-F5F6-4FA9-9729-4D7F3738B1EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117302), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:29.103 4076 3051113161656 MainThread DEBUG os_win._utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.71319580078125 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:29.139 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-a53ceb25-2e18-446b-b68f-d3a9d9f344f8 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:29.139 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-a53ceb25-2e18-446b-b68f-d3a9d9f344f8 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:29.186 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-a53ceb25-2e18-446b-b68f-d3a9d9f344f8 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:29.226 4076 3051113161832 MainThread INFO nova.compute.manager [req-a53ceb25-2e18-446b-b68f-d3a9d9f344f8 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:29.369 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Received event network-changed-20f19fa9-4c3e-43e8-8e22-6987b697fd3b external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:29.369 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Refreshing instance network info cache due to event network-changed-20f19fa9-4c3e-43e8-8e22-6987b697fd3b. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:29.369 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:29.369 4076 3051113162008 MainThread DEBUG nova.network.neutron [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Refreshing network info cache for port 20f19fa9-4c3e-43e8-8e22-6987b697fd3b _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:29.492 4076 3051113161480 MainThread INFO nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Booting with volume 11b5cce0-1a22-4820-b4b0-bcc0b921fc84 at /dev/sdb 2021-09-10 16:58:29.616 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:58:29.616 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:58:29.616 4076 3051113161480 MainThread DEBUG nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating existing volume attachment record: 3cc504a9-711e-4115-932b-2d1bac39ade8 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:58:29.654 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:58:29.754 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'BD679126-E4D7-4C7C-883C-C8F37DE1324A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=106433), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:29.796 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:29.796 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:29.796 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:29.814 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:29.814 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:29.839 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-0968f9db-2044-4055-b7a1-b65510fdcd97 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:29.839 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-0968f9db-2044-4055-b7a1-b65510fdcd97 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:29.889 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-0968f9db-2044-4055-b7a1-b65510fdcd97 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:29.913 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BD679126-E4D7-4C7C-883C-C8F37DE1324A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120759), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:29.913 4076 3051113163064 MainThread DEBUG os_win._utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.468489170074463 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:29.979 4076 3051113164648 MainThread INFO nova.compute.manager [req-0968f9db-2044-4055-b7a1-b65510fdcd97 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:29.979 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully created port: 7210f512-68cf-4fad-b1ca-29f66e80f530 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:30.164 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '2B1DE14C-4D43-47A4-B43A-9E3975276A58', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15403), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:30.305 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:30.305 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:30.305 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:30.305 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:30.305 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2B1DE14C-4D43-47A4-B43A-9E3975276A58', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120050), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:30.305 4076 3051113161656 MainThread DEBUG os_win._utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.9155592918395996 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:30.348 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-26630ef4-e446-4d94-a4a5-f10ff857ee74 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:30.348 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-26630ef4-e446-4d94-a4a5-f10ff857ee74 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:30.405 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-26630ef4-e446-4d94-a4a5-f10ff857ee74 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:30.432 4076 3051113160776 MainThread INFO nova.compute.manager [req-26630ef4-e446-4d94-a4a5-f10ff857ee74 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:30.432 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:30.432 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:30.432 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:30.464 4076 3051113160952 MainThread DEBUG os_win.utils.compute.vmutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed vm state of instance-00000037 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:30.464 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed state of VM instance-00000037 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:30.507 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'B702750F-91CD-4368-A272-58AC75F16CF3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20341), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:30.633 4076 3051113160952 MainThread DEBUG os_win.utils.jobutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job succeeded: {'InstanceID': 'B702750F-91CD-4368-A272-58AC75F16CF3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29980), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:30.633 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:30.633 4076 3051113160952 MainThread DEBUG os_win.utils.pathutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Removing directory: C:\openstack\instances\instance-00000037 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:30.633 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:30.633 4076 3051113160952 MainThread ERROR nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Traceback (most recent call last): 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(*args, **kwargs) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] admin_password, network_info, block_device_info) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(self, *args, **kwds) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] should_plug_vifs=False) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'req_state': req_state}) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] time.sleep(sleep_time) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return f(*args, **kwargs) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return self._wait_for_job(job_path) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] error_desc=err_desc) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During handling of the above exception, another exception occurred: 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Traceback (most recent call last): 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] yield resources 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] accel_info=accel_info) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(*args, **kwargs) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] admin_password, network_info, block_device_info) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(self, *args, **kwds) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] should_plug_vifs=False) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'req_state': req_state}) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] time.sleep(sleep_time) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return f(*args, **kwargs) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return self._wait_for_job(job_path) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] error_desc=err_desc) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:30.633 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:30.633 4076 3051113160952 MainThread INFO nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Terminating instance 2021-09-10 16:58:30.633 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:30.633 4076 3051113160952 MainThread INFO nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Got request to destroy instance 2021-09-10 16:58:30.656 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:30.667 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:30.667 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000037" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:30.667 4076 3051113160952 MainThread INFO nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:30.667 4076 3051113160952 MainThread DEBUG nova.compute.claims [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:30.667 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:30.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:30.784 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:58:30.873 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.hostops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:30.890 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.hostops [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:30.935 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '723A0474-62B5-4522-BE1C-EBF989B9C529', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17710), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:30.977 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:30.977 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:58:31.089 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:31.089 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:31.089 4076 3051113162536 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:31.089 4076 3051113162536 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:31.089 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '723A0474-62B5-4522-BE1C-EBF989B9C529', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118091), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:31.089 4076 3051113163064 MainThread DEBUG os_win._utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6442105770111084 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:31.105 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-b025f4ae-5029-4804-87ff-a883525c2647 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:31.105 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-b025f4ae-5029-4804-87ff-a883525c2647 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:31.105 4076 3051113162008 MainThread DEBUG nova.network.neutron [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Updated VIF entry in instance network info cache for port 20f19fa9-4c3e-43e8-8e22-6987b697fd3b. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:58:31.105 4076 3051113162008 MainThread DEBUG nova.network.neutron [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Updating instance_info_cache with network_info: [{"id": "20f19fa9-4c3e-43e8-8e22-6987b697fd3b", "address": "fa:16:3e:b4:44:ec", "network": {"id": "6b3d4f05-5619-41ec-8def-1bd17c9bcd14", "bridge": null, "label": "tempest-ServerRescueTestJSON-1067413489-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a13940e712fe464c8a1b2c3732982edb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap20f19fa9-4c", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:31.120 4076 3051113160952 MainThread DEBUG nova.compute.provider_tree [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.scheduler.client.report [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:31.152 4076 3051113160952 MainThread ERROR nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Traceback (most recent call last): 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(*args, **kwargs) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] admin_password, network_info, block_device_info) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(self, *args, **kwds) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] should_plug_vifs=False) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'req_state': req_state}) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] time.sleep(sleep_time) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return f(*args, **kwargs) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return self._wait_for_job(job_path) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] error_desc=err_desc) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] During handling of the above exception, another exception occurred: 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Traceback (most recent call last): 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] accel_info=accel_info) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(*args, **kwargs) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] admin_password, network_info, block_device_info) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return function(self, *args, **kwds) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] should_plug_vifs=False) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'req_state': req_state}) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] time.sleep(sleep_time) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self.force_reraise() 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] raise self.value 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return f(*args, **kwargs) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] return self._wait_for_job(job_path) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] error_desc=err_desc) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. 2021-09-10 16:58:31.152 4076 3051113160952 MainThread TRACE nova.compute.manager [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.compute.utils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Build of instance 082d9c5d-01d4-43af-8493-80ba655501fd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000037' failed to start. Not enough memory in the system to start the virtual machine instance-00000037. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000037' failed to start. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Not enough memory in the system to start the virtual machine instance-00000037 with ram size 1024 megabytes. (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) 'instance-00000037' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CD065D42-35B1-4A7A-947D-866C4098026C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:31.152 4076 3051113160952 MainThread DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:31.170 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-2906301b-d035-4fa6-be26-13a355e737e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:31.179 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-b025f4ae-5029-4804-87ff-a883525c2647 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:31.225 4076 3051113162536 MainThread INFO nova.compute.manager [req-b025f4ae-5029-4804-87ff-a883525c2647 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:31.351 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '158E0F9F-614D-4FFD-AB18-0196FEFEB80A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15485), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:31.460 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:31.460 4076 3051093170488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:31.460 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:31.460 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:31.492 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '158E0F9F-614D-4FFD-AB18-0196FEFEB80A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114487), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:31.492 4076 3051113161656 MainThread DEBUG os_win._utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.102976560592651 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:31.524 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-d50541ec-c646-4baf-8ee6-4c9517cf79c0 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:31.524 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-d50541ec-c646-4baf-8ee6-4c9517cf79c0 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:31.570 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-d50541ec-c646-4baf-8ee6-4c9517cf79c0 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:31.601 4076 3051093171368 MainThread INFO nova.compute.manager [req-d50541ec-c646-4baf-8ee6-4c9517cf79c0 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:32.141 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '3478EB46-38AF-442A-A7D7-4ED167AA42F0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14503), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:32.267 4076 3051113162536 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:32.267 4076 3051113162536 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:32.280 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:32.280 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:32.289 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3478EB46-38AF-442A-A7D7-4ED167AA42F0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112635), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:32.289 4076 3051113163064 MainThread DEBUG os_win._utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8442397117614746 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:32.308 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-f7c7c6d3-3ac1-4eb0-be5c-d06c98c012b7 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:32.308 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-f7c7c6d3-3ac1-4eb0-be5c-d06c98c012b7 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:32.336 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-f7c7c6d3-3ac1-4eb0-be5c-d06c98c012b7 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:32.403 4076 3051113162536 MainThread INFO nova.compute.manager [req-f7c7c6d3-3ac1-4eb0-be5c-d06c98c012b7 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:32.509 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'AE290F2D-D1EF-4E9D-982F-A560174CB020', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15008), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:32.663 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:32.663 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:32.666 4076 3051113162536 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:32.666 4076 3051113162536 MainThread INFO nova.compute.manager [-] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] VM Stopped (Lifecycle Event) 2021-09-10 16:58:32.666 4076 3051113161656 MainThread ERROR os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000039\' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD05123122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000039\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)instance-00000039F406A07D-3B80-4893-AE10-ED1E2D35FCD0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AE290F2D-D1EF-4E9D-982F-A560174CB020', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113963), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\nNot enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)\n\n'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0)", 'ErrorSummaryDescription': "'instance-00000039' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000039.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:32.666 4076 3051113161656 MainThread ERROR nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Failed to change vm state of instance-00000039 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. 2021-09-10 16:58:32.666 4076 3051113161656 MainThread INFO nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Got request to destroy instance 2021-09-10 16:58:32.699 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-45c6aba9-34a4-452d-aa9e-ce50b3ff5f91 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:32.699 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-45c6aba9-34a4-452d-aa9e-ce50b3ff5f91 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:32.721 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:32.726 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:32.726 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:58:32.726 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:32.726 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:32.750 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-45c6aba9-34a4-452d-aa9e-ce50b3ff5f91 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:32.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:32.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:32.790 4076 3051113162536 MainThread INFO nova.compute.manager [req-45c6aba9-34a4-452d-aa9e-ce50b3ff5f91 - - - - -] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:32.790 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully created port: bb905c29-b72e-4c4c-b524-e08c9df4bbd8 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:33.179 4076 3051113160952 MainThread DEBUG nova.network.neutron [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:33.206 4076 3051113160952 MainThread INFO nova.compute.manager [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 082d9c5d-01d4-43af-8493-80ba655501fd] Took 2.06 seconds to deallocate network for instance. 2021-09-10 16:58:33.342 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': 'F5CD10C2-3CF2-42A8-A80F-DFF65266F6EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15703), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:33.492 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F5CD10C2-3CF2-42A8-A80F-DFF65266F6EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115361), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:33.492 4076 3051113163064 MainThread DEBUG os_win._utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.0476603507995605 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:33.492 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:33.492 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:33.492 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:33.492 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:33.525 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-b1208eb9-b6db-43be-9344-bfea4971cb61 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:33.525 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-b1208eb9-b6db-43be-9344-bfea4971cb61 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:33.551 4076 3051113160952 MainThread INFO nova.scheduler.client.report [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Deleted allocations for instance 082d9c5d-01d4-43af-8493-80ba655501fd 2021-09-10 16:58:33.572 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-b1208eb9-b6db-43be-9344-bfea4971cb61 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:33.613 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-37148690-7891-402c-9de8-abf1d53bc074 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "082d9c5d-01d4-43af-8493-80ba655501fd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.969s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:33.648 4076 3051113162008 MainThread INFO nova.compute.manager [req-b1208eb9-b6db-43be-9344-bfea4971cb61 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:33.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:34.523 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '36CD5259-D7DC-41DA-8DF5-D94C70B4E19F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15570), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:34.556 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:34.556 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.556 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:34.579 4076 3051113161656 MainThread DEBUG os_win.utils.compute.vmutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed vm state of instance-00000039 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:34.592 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed state of VM instance-00000039 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:34.634 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '8FD2DD41-3652-41D0-B838-2FAAC67648C6', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17750), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:34.666 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:34.666 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] VM Stopped (Lifecycle Event) 2021-09-10 16:58:34.666 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.666 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:34.666 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000038\' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB74830210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000038\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)instance-00000038AE3557D8-CB0C-47B3-A82F-6001AB748302%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '36CD5259-D7DC-41DA-8DF5-D94C70B4E19F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115460), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\nNot enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)\n\n'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302)", 'ErrorSummaryDescription': "'instance-00000038' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000038.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:34.666 4076 3051113163064 MainThread ERROR nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Failed to change vm state of instance-00000038 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. 2021-09-10 16:58:34.666 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Got request to destroy instance 2021-09-10 16:58:34.707 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-471614eb-d4d6-49bd-9b02-f2f9a1168c70 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:34.707 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-471614eb-d4d6-49bd-9b02-f2f9a1168c70 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:34.745 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-471614eb-d4d6-49bd-9b02-f2f9a1168c70 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:34.745 4076 3051113161656 MainThread DEBUG os_win.utils.jobutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job succeeded: {'InstanceID': '8FD2DD41-3652-41D0-B838-2FAAC67648C6', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=36394), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:34.745 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.745 4076 3051113161656 MainThread DEBUG os_win.utils.pathutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Removing directory: C:\openstack\instances\instance-00000039 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:34.766 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:34.766 4076 3051113161656 MainThread ERROR nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Traceback (most recent call last): 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(*args, **kwargs) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] admin_password, network_info, block_device_info) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(self, *args, **kwds) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] should_plug_vifs=False) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'req_state': req_state}) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] time.sleep(sleep_time) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return f(*args, **kwargs) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return self._wait_for_job(job_path) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] error_desc=err_desc) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During handling of the above exception, another exception occurred: 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Traceback (most recent call last): 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] yield resources 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] accel_info=accel_info) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(*args, **kwargs) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] admin_password, network_info, block_device_info) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(self, *args, **kwds) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] should_plug_vifs=False) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'req_state': req_state}) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] time.sleep(sleep_time) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return f(*args, **kwargs) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return self._wait_for_job(job_path) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] error_desc=err_desc) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:34.766 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:34.766 4076 3051113161656 MainThread INFO nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Terminating instance 2021-09-10 16:58:34.766 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:34.766 4076 3051113161656 MainThread INFO nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Got request to destroy instance 2021-09-10 16:58:34.766 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-3852c0e9-f4c0-4635-9d9e-99e8ac86f170 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Received event network-vif-plugged-2ab80196-8434-4424-b44a-f47ed0cf89ad external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:34.780 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-3852c0e9-f4c0-4635-9d9e-99e8ac86f170 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.780 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-3852c0e9-f4c0-4635-9d9e-99e8ac86f170 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:34.780 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-3852c0e9-f4c0-4635-9d9e-99e8ac86f170 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Processing event network-vif-plugged-2ab80196-8434-4424-b44a-f47ed0cf89ad _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:34.793 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:34.793 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.793 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000039" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:34.793 4076 3051113161656 MainThread INFO nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:34.793 4076 3051113161656 MainThread DEBUG nova.compute.claims [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:34.793 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:34.804 4076 3051113162008 MainThread INFO nova.compute.manager [req-471614eb-d4d6-49bd-9b02-f2f9a1168c70 - - - - -] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:34.820 4076 3051113161304 MainThread INFO nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Using config drive for instance 2021-09-10 16:58:35.094 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.hostops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:35.094 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully created port: b083b498-e447-45f8-9086-ac65ceea526a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:58:35.117 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.hostops [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:35.306 4076 3051113161304 MainThread INFO nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Creating config drive at C:\openstack\instances\instance-0000003a\configdrive.iso 2021-09-10 16:58:35.338 4076 3051113161304 MainThread DEBUG oslo_concurrency.processutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpeii2zuuk execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:35.521 4076 3051113161656 MainThread DEBUG nova.compute.provider_tree [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:35.535 4076 3051113161656 MainThread DEBUG nova.scheduler.client.report [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.750s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:35.542 4076 3051113161656 MainThread ERROR nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Traceback (most recent call last): 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(*args, **kwargs) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] admin_password, network_info, block_device_info) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(self, *args, **kwds) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] should_plug_vifs=False) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'req_state': req_state}) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] time.sleep(sleep_time) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return f(*args, **kwargs) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return self._wait_for_job(job_path) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] error_desc=err_desc) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] During handling of the above exception, another exception occurred: 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Traceback (most recent call last): 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] accel_info=accel_info) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(*args, **kwargs) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] admin_password, network_info, block_device_info) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return function(self, *args, **kwds) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] should_plug_vifs=False) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'req_state': req_state}) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] time.sleep(sleep_time) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self.force_reraise() 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] raise self.value 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return f(*args, **kwargs) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] return self._wait_for_job(job_path) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] error_desc=err_desc) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. 2021-09-10 16:58:35.542 4076 3051113161656 MainThread TRACE nova.compute.manager [instance: 832a0158-f115-480e-a24e-78506e51b9f6] 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.compute.utils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Build of instance 832a0158-f115-480e-a24e-78506e51b9f6 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000039' failed to start. Not enough memory in the system to start the virtual machine instance-00000039. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000039' failed to start. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Not enough memory in the system to start the virtual machine instance-00000039 with ram size 512 megabytes. (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) 'instance-00000039' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F406A07D-3B80-4893-AE10-ED1E2D35FCD0) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:35.542 4076 3051113161656 MainThread DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:35.651 4076 3051113161304 MainThread DEBUG oslo_concurrency.processutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpeii2zuuk" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:35.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:35.807 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:35.821 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:35.821 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:35.821 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:58:35.821 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:35.845 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:36.070 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2220MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:58:36.070 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:36.070 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Setting boot order for instance: instance-0000003a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:36.124 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:36.150 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '7DD27172-029B-4AA3-9CD9-5EAE57ABD15F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17104), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:36.314 4076 3051113162536 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:36.321 4076 3051113162536 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:36.321 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:36.321 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:36.321 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7DD27172-029B-4AA3-9CD9-5EAE57ABD15F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114282), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:36.321 4076 3051113161304 MainThread DEBUG os_win._utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1963510513305664 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:36.348 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-bf428fc8-b70a-4bb5-91c8-71b5c9d2a2fb - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:36.348 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-bf428fc8-b70a-4bb5-91c8-71b5c9d2a2fb - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:36.400 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-bf428fc8-b70a-4bb5-91c8-71b5c9d2a2fb - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.422 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051113162536 MainThread INFO nova.compute.manager [req-bf428fc8-b70a-4bb5-91c8-71b5c9d2a2fb - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 832a0158-f115-480e-a24e-78506e51b9f6 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance c7fdc9fc-d133-4f12-ab12-ff417c249240 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 1f6d373b-5624-411c-9545-511d98e94f86 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance b0745679-4000-4aff-bd0a-29ded4711f24 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 16 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:58:36.452 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=16896MB phys_disk=79GB used_disk=15GB total_vcpus=6 used_vcpus=16 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:58:36.503 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:36.524 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:36.738 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:36.743 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:36.743 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:58:36.743 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.672s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:36.825 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:36.825 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:36.825 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:36.850 4076 3051113163064 MainThread DEBUG os_win.utils.compute.vmutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed vm state of instance-00000038 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:36.850 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Successfully changed state of VM instance-00000038 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:36.903 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Waiting for WMI job: {'InstanceID': '2B402207-4954-45A3-AAD3-31BC009D2A70', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13909), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:36.962 4076 3051113161480 MainThread INFO nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Booting with volume f0f70f22-4412-4d14-80e3-17ea362502a8 at /dev/sdc 2021-09-10 16:58:37.007 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] WMI job succeeded: {'InstanceID': '2B402207-4954-45A3-AAD3-31BC009D2A70', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29354), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:37.007 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:37.007 4076 3051113163064 MainThread DEBUG os_win.utils.pathutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Removing directory: C:\openstack\instances\instance-00000038 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:37.007 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:37.007 4076 3051113163064 MainThread ERROR nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Traceback (most recent call last): 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(*args, **kwargs) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] admin_password, network_info, block_device_info) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(self, *args, **kwds) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] should_plug_vifs=False) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'req_state': req_state}) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] time.sleep(sleep_time) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return f(*args, **kwargs) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return self._wait_for_job(job_path) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] error_desc=err_desc) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During handling of the above exception, another exception occurred: 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Traceback (most recent call last): 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] yield resources 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] accel_info=accel_info) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(*args, **kwargs) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] admin_password, network_info, block_device_info) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(self, *args, **kwds) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] should_plug_vifs=False) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'req_state': req_state}) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] time.sleep(sleep_time) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return f(*args, **kwargs) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return self._wait_for_job(job_path) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] error_desc=err_desc) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.007 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.026 4076 3051113163064 MainThread INFO nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Terminating instance 2021-09-10 16:58:37.026 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:37.026 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Got request to destroy instance 2021-09-10 16:58:37.045 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:58:37.045 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:37.045 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:37.045 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "instance-00000038" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:37.045 4076 3051113163064 MainThread INFO nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:37.045 4076 3051113163064 MainThread DEBUG nova.compute.claims [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:37.045 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:37.059 4076 3051113161480 MainThread DEBUG os_brick.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== get_connector_properties: return (14ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:58:37.059 4076 3051113161480 MainThread DEBUG nova.virt.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating existing volume attachment record: bad5044e-ab15-4c3f-9340-6c3fd5ccbbe6 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:58:37.195 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:37.226 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:37.335 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-a9aa8582-419f-4786-bee2-8dfc55516343 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Received event network-vif-plugged-20f19fa9-4c3e-43e8-8e22-6987b697fd3b external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:37.335 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-a9aa8582-419f-4786-bee2-8dfc55516343 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:37.335 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-a9aa8582-419f-4786-bee2-8dfc55516343 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:37.335 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-a9aa8582-419f-4786-bee2-8dfc55516343 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Processing event network-vif-plugged-20f19fa9-4c3e-43e8-8e22-6987b697fd3b _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:58:37.335 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'C9EAB7E1-7827-4D8C-ABBF-9CFAF156CC67', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15385), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:37.404 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Using config drive for instance 2021-09-10 16:58:37.444 4076 3051113163064 MainThread DEBUG nova.compute.provider_tree [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.scheduler.client.report [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:37.463 4076 3051113163064 MainThread ERROR nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Traceback (most recent call last): 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(*args, **kwargs) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] admin_password, network_info, block_device_info) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(self, *args, **kwds) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] should_plug_vifs=False) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'req_state': req_state}) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] time.sleep(sleep_time) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return f(*args, **kwargs) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return self._wait_for_job(job_path) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] error_desc=err_desc) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] During handling of the above exception, another exception occurred: 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Traceback (most recent call last): 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] accel_info=accel_info) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(*args, **kwargs) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] admin_password, network_info, block_device_info) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return function(self, *args, **kwds) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] should_plug_vifs=False) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'req_state': req_state}) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] time.sleep(sleep_time) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self.force_reraise() 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] raise self.value 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return f(*args, **kwargs) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] return self._wait_for_job(job_path) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] error_desc=err_desc) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. 2021-09-10 16:58:37.463 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.compute.utils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Build of instance c7fdc9fc-d133-4f12-ab12-ff417c249240 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000038' failed to start. Not enough memory in the system to start the virtual machine instance-00000038. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000038' failed to start. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Not enough memory in the system to start the virtual machine instance-00000038 with ram size 1024 megabytes. (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) 'instance-00000038' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AE3557D8-CB0C-47B3-A82F-6001AB748302) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:37.463 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:37.491 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:37.496 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:37.496 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:37.496 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:37.496 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C9EAB7E1-7827-4D8C-ABBF-9CFAF156CC67', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114382), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:37.496 4076 3051113161304 MainThread DEBUG os_win._utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.371354579925537 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:37.515 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-acc12fad-bab6-4094-a289-511b25576155 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:37.515 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-acc12fad-bab6-4094-a289-511b25576155 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:37.562 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-acc12fad-bab6-4094-a289-511b25576155 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:37.585 4076 3051093171016 MainThread INFO nova.compute.manager [req-acc12fad-bab6-4094-a289-511b25576155 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:37.742 4076 3051113161656 MainThread DEBUG nova.network.neutron [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:37.764 4076 3051113161656 MainThread INFO nova.compute.manager [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: 832a0158-f115-480e-a24e-78506e51b9f6] Took 2.22 seconds to deallocate network for instance. 2021-09-10 16:58:38.026 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: 4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:38.154 4076 3051113161656 MainThread INFO nova.scheduler.client.report [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Deleted allocations for instance 832a0158-f115-480e-a24e-78506e51b9f6 2021-09-10 16:58:38.189 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Creating config drive at C:\openstack\instances\instance-0000003c\configdrive.iso 2021-09-10 16:58:38.225 4076 3051113163768 MainThread DEBUG oslo_concurrency.processutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpxaa1phyl execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:58:38.369 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-8da4dc52-4693-4bdc-ba36-0aa8f3570d94 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "832a0158-f115-480e-a24e-78506e51b9f6" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.688s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:38.516 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'B999750D-C1BD-4897-B42C-37B772CE4CDE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15940), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:38.526 4076 3051113163768 MainThread DEBUG oslo_concurrency.processutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpxaa1phyl" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:58:38.649 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:38.649 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:38.649 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:38.649 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:38.649 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B999750D-C1BD-4897-B42C-37B772CE4CDE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113621), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:38.675 4076 3051113161304 MainThread DEBUG os_win._utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.551149845123291 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:38.694 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-1765e874-3b2b-4b93-be88-88ebbb13c9d5 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:38.694 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-1765e874-3b2b-4b93-be88-88ebbb13c9d5 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:38.746 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-1765e874-3b2b-4b93-be88-88ebbb13c9d5 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:38.766 4076 3051113164120 MainThread INFO nova.compute.manager [req-1765e874-3b2b-4b93-be88-88ebbb13c9d5 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:38.925 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Setting boot order for instance: instance-0000003c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:58:38.978 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:58:39.006 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': '06C1357F-E6A9-40BD-BE49-3E6291F7267E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15930), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:39.137 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:39.137 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:39.137 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:39.137 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:39.157 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '06C1357F-E6A9-40BD-BE49-3E6291F7267E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:39.157 4076 3051113163768 MainThread DEBUG os_win._utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1786036491394043 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:39.174 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-bffbc9f8-695e-4654-87e9-fee01f655071 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:39.174 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-bffbc9f8-695e-4654-87e9-fee01f655071 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:39.222 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-bffbc9f8-695e-4654-87e9-fee01f655071 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:39.253 4076 3051113164120 MainThread INFO nova.compute.manager [req-bffbc9f8-695e-4654-87e9-fee01f655071 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:39.695 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'B2DF2A90-D1DA-4B5E-AC36-A01AF8BEC212', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15039), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:39.722 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:58:39.722 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:58:39.839 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:39.839 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:39.839 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:39.839 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:39.839 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B2DF2A90-D1DA-4B5E-AC36-A01AF8BEC212', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111040), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:39.839 4076 3051113161304 MainThread DEBUG os_win._utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.715174913406372 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:39.839 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-061a686d-8f56-4d28-8102-d2aa8f2bb702 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:39.866 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-061a686d-8f56-4d28-8102-d2aa8f2bb702 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:39.898 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-061a686d-8f56-4d28-8102-d2aa8f2bb702 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:39.979 4076 3051113164120 MainThread INFO nova.compute.manager [req-061a686d-8f56-4d28-8102-d2aa8f2bb702 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:40.187 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': '3FD411CD-A7EE-4747-BA9D-E96A24D22132', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14466), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:40.321 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:40.321 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:40.338 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:40.338 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:40.353 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3FD411CD-A7EE-4747-BA9D-E96A24D22132', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109750), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:40.353 4076 3051113163768 MainThread DEBUG os_win._utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3746998310089111 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:40.353 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-1ab9949e-c3f8-4e32-baed-0e53dc7f3b36 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:40.353 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-1ab9949e-c3f8-4e32-baed-0e53dc7f3b36 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:40.398 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-1ab9949e-c3f8-4e32-baed-0e53dc7f3b36 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:40.444 4076 3051113164120 MainThread INFO nova.compute.manager [req-1ab9949e-c3f8-4e32-baed-0e53dc7f3b36 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:40.455 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:40.455 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:40.455 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:40.455 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port 4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:40.582 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:40.862 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:40.862 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '17A325F6-7C8E-4E52-9131-A878B605EF2A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14693), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:40.883 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-04d7ed01-d883-432c-966b-6e0aee6d823d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:41.007 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:41.007 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:41.007 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:41.007 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:41.012 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '17A325F6-7C8E-4E52-9131-A878B605EF2A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110993), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:41.012 4076 3051113161304 MainThread DEBUG os_win._utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.887577056884766 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:41.038 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-af97fdd0-d743-4cdf-90e4-3ba7e92d8461 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:41.038 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-af97fdd0-d743-4cdf-90e4-3ba7e92d8461 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:41.079 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-af97fdd0-d743-4cdf-90e4-3ba7e92d8461 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:41.112 4076 3051113164120 MainThread INFO nova.compute.manager [req-af97fdd0-d743-4cdf-90e4-3ba7e92d8461 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:41.378 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': 'C7A160C0-5C02-43D4-94A8-3325CE0327B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15491), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:41.494 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:41.522 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:41.522 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:41.525 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:41.525 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C7A160C0-5C02-43D4-94A8-3325CE0327B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112012), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:41.525 4076 3051113163768 MainThread DEBUG os_win._utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5472092628479004 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:41.566 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-5c3244e2-4d62-4be3-840b-90fb016f3fcb - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:41.566 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-5c3244e2-4d62-4be3-840b-90fb016f3fcb - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:41.602 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-5c3244e2-4d62-4be3-840b-90fb016f3fcb - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:41.757 4076 3051113164120 MainThread INFO nova.compute.manager [req-5c3244e2-4d62-4be3-840b-90fb016f3fcb - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:41.949 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:41.972 4076 3051113163064 MainThread INFO nova.compute.manager [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] [instance: c7fdc9fc-d133-4f12-ab12-ff417c249240] Took 4.52 seconds to deallocate network for instance. 2021-09-10 16:58:42.038 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '1E572ECE-9161-49FA-ADC1-F9B0CC3E2FA2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15122), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:42.194 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:42.194 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] VM Stopped (Lifecycle Event) 2021-09-10 16:58:42.194 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:42.194 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:42.194 4076 3051113161304 MainThread ERROR os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003a\' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-98393794531710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)instance-0000003aDEE8B75B-FC7A-4F01-B810-983937945317%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1E572ECE-9161-49FA-ADC1-F9B0CC3E2FA2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117251), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\nNot enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)\n\n'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317)", 'ErrorSummaryDescription': "'instance-0000003a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:42.194 4076 3051113161304 MainThread ERROR nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Failed to change vm state of instance-0000003a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. 2021-09-10 16:58:42.194 4076 3051113161304 MainThread INFO nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Got request to destroy instance 2021-09-10 16:58:42.207 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-4e2f30c5-d763-4a6e-ae43-a0d8d275f114 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:42.207 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-4e2f30c5-d763-4a6e-ae43-a0d8d275f114 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:42.224 4076 3051113163064 MainThread INFO nova.scheduler.client.report [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Deleted allocations for instance c7fdc9fc-d133-4f12-ab12-ff417c249240 2021-09-10 16:58:42.274 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-63c3c131-0966-429d-b576-33bfc4986f37 e9b8fdd82fe341dc8eeee9f6dea5f66f cc8daf6ce3114d6aa694f16669c8ed20 - default default] Lock "c7fdc9fc-d133-4f12-ab12-ff417c249240" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 40.281s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:42.274 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-4e2f30c5-d763-4a6e-ae43-a0d8d275f114 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:42.304 4076 3051113164120 MainThread INFO nova.compute.manager [req-4e2f30c5-d763-4a6e-ae43-a0d8d275f114 - - - - -] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:42.544 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': '541F6AEA-1357-44E7-BFD1-0F4E453DDB5E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16668), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:42.679 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:42.679 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:42.679 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:42.679 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:42.679 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '541F6AEA-1357-44E7-BFD1-0F4E453DDB5E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112529), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:42.679 4076 3051113163768 MainThread DEBUG os_win._utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.701129913330078 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:42.710 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-03db986e-94b9-4974-b736-54f3a59e9a0d - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:42.710 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-03db986e-94b9-4974-b736-54f3a59e9a0d - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:42.747 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-03db986e-94b9-4974-b736-54f3a59e9a0d - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:42.770 4076 3051113164120 MainThread INFO nova.compute.manager [req-03db986e-94b9-4974-b736-54f3a59e9a0d - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:42.994 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: b922a519-3efc-4024-abfb-de5c20d9d578 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:43.242 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:58:43.242 4076 3051113161480 MainThread INFO nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Spawning new instance 2021-09-10 16:58:43.260 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:43.260 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:43.260 4076 3051113161480 MainThread DEBUG nova.block_device [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] block_device_list ['sda', 'sdb', 'sdc'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:58:43.722 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': 'E21721C0-8240-457A-8479-ABA06912AA27', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16787), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:43.835 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:43.835 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:43.835 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:43.835 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:43.873 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E21721C0-8240-457A-8479-ABA06912AA27', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110984), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:43.873 4076 3051113163768 MainThread DEBUG os_win._utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.894530773162842 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:58:43.873 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-c53a2d9b-051e-401b-8671-2da5ee511ad1 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:43.873 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-c53a2d9b-051e-401b-8671-2da5ee511ad1 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:43.934 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-c53a2d9b-051e-401b-8671-2da5ee511ad1 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:43.948 4076 3051113162008 MainThread INFO nova.compute.manager [req-c53a2d9b-051e-401b-8671-2da5ee511ad1 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:44.050 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:44.069 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:44.069 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:44.085 4076 3051113161304 MainThread DEBUG os_win.utils.compute.vmutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed vm state of instance-0000003a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:44.085 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed state of VM instance-0000003a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:44.132 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'CD058C5E-B397-45B2-AAEC-3F3E19F5EF2C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14869), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:44.257 4076 3051113161304 MainThread DEBUG os_win.utils.jobutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job succeeded: {'InstanceID': 'CD058C5E-B397-45B2-AAEC-3F3E19F5EF2C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29160), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:44.257 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:44.257 4076 3051113161304 MainThread DEBUG os_win.utils.pathutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Removing directory: C:\openstack\instances\instance-0000003a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:44.257 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:44.257 4076 3051113161304 MainThread ERROR nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Traceback (most recent call last): 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(*args, **kwargs) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] admin_password, network_info, block_device_info) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(self, *args, **kwds) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] should_plug_vifs=False) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'req_state': req_state}) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] time.sleep(sleep_time) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return f(*args, **kwargs) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return self._wait_for_job(job_path) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] error_desc=err_desc) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During handling of the above exception, another exception occurred: 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Traceback (most recent call last): 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] yield resources 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] accel_info=accel_info) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(*args, **kwargs) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] admin_password, network_info, block_device_info) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(self, *args, **kwds) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] should_plug_vifs=False) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'req_state': req_state}) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] time.sleep(sleep_time) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return f(*args, **kwargs) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return self._wait_for_job(job_path) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] error_desc=err_desc) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.257 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.257 4076 3051113161304 MainThread INFO nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Terminating instance 2021-09-10 16:58:44.257 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:44.257 4076 3051113161304 MainThread INFO nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Got request to destroy instance 2021-09-10 16:58:44.283 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:44.283 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:44.283 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-0000003a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:44.283 4076 3051113161304 MainThread INFO nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:44.283 4076 3051113161304 MainThread DEBUG nova.compute.claims [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:44.283 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:44.540 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.hostops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:44.540 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.hostops [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:44.757 4076 3051113161304 MainThread DEBUG nova.compute.provider_tree [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.scheduler.client.report [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:44.796 4076 3051113161304 MainThread ERROR nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Traceback (most recent call last): 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(*args, **kwargs) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] admin_password, network_info, block_device_info) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(self, *args, **kwds) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] should_plug_vifs=False) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'req_state': req_state}) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] time.sleep(sleep_time) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return f(*args, **kwargs) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return self._wait_for_job(job_path) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] error_desc=err_desc) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] During handling of the above exception, another exception occurred: 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Traceback (most recent call last): 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] accel_info=accel_info) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(*args, **kwargs) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] admin_password, network_info, block_device_info) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return function(self, *args, **kwds) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] should_plug_vifs=False) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'req_state': req_state}) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] time.sleep(sleep_time) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self.force_reraise() 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] raise self.value 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return f(*args, **kwargs) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] return self._wait_for_job(job_path) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] error_desc=err_desc) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. 2021-09-10 16:58:44.796 4076 3051113161304 MainThread TRACE nova.compute.manager [instance: 1f6d373b-5624-411c-9545-511d98e94f86] 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.compute.utils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Build of instance 1f6d373b-5624-411c-9545-511d98e94f86 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003a' failed to start. Not enough memory in the system to start the virtual machine instance-0000003a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003a' failed to start. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Not enough memory in the system to start the virtual machine instance-0000003a with ram size 1024 megabytes. (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) 'instance-0000003a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DEE8B75B-FC7A-4F01-B810-983937945317) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:44.796 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:44.887 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': '0D46F968-5036-4C66-BAAF-8D1118396E35', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15308), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:45.026 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:58:45.026 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] VM Stopped (Lifecycle Event) 2021-09-10 16:58:45.026 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:45.026 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:45.026 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003c\' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)instance-0000003c85FABA55-A2FD-4B6D-9800-A973AC530C2F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0D46F968-5036-4C66-BAAF-8D1118396E35', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110245), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 58, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 58, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\nNot enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)\n\n'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F)", 'ErrorSummaryDescription': "'instance-0000003c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:58:45.026 4076 3051113163768 MainThread ERROR nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Failed to change vm state of instance-0000003c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. 2021-09-10 16:58:45.026 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Got request to destroy instance 2021-09-10 16:58:45.041 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-7a8d3d5f-d656-48e8-b9aa-9083113ff8f0 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:58:45.041 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8d3d5f-d656-48e8-b9aa-9083113ff8f0 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:58:45.109 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-7a8d3d5f-d656-48e8-b9aa-9083113ff8f0 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:58:45.139 4076 3051093170312 MainThread INFO nova.compute.manager [req-7a8d3d5f-d656-48e8-b9aa-9083113ff8f0 - - - - -] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:58:46.841 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:58:46.841 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:46.841 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:46.866 4076 3051113163768 MainThread DEBUG os_win.utils.compute.vmutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Successfully changed vm state of instance-0000003c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:58:46.866 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Successfully changed state of VM instance-0000003c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:58:46.906 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Waiting for WMI job: {'InstanceID': '51A32069-118C-4569-A026-CBF903FF5381', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14005), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:58:47.021 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] WMI job succeeded: {'InstanceID': '51A32069-118C-4569-A026-CBF903FF5381', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28674), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:58:47.038 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG os_win.utils.pathutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Removing directory: C:\openstack\instances\instance-0000003c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:47.039 4076 3051113163768 MainThread ERROR nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Traceback (most recent call last): 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(*args, **kwargs) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] admin_password, network_info, block_device_info) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(self, *args, **kwds) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] should_plug_vifs=False) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'req_state': req_state}) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] time.sleep(sleep_time) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return f(*args, **kwargs) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return self._wait_for_job(job_path) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] error_desc=err_desc) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During handling of the above exception, another exception occurred: 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Traceback (most recent call last): 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] yield resources 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] accel_info=accel_info) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(*args, **kwargs) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] admin_password, network_info, block_device_info) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(self, *args, **kwds) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] should_plug_vifs=False) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'req_state': req_state}) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] time.sleep(sleep_time) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return f(*args, **kwargs) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return self._wait_for_job(job_path) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] error_desc=err_desc) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.039 4076 3051113163768 MainThread INFO nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Terminating instance 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:58:47.039 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Got request to destroy instance 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "instance-0000003c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:47.039 4076 3051113163768 MainThread INFO nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG nova.compute.claims [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:58:47.039 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:58:47.168 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: 46dd9dca-3150-45cc-9a35-bc1230d1380e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:47.284 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:58:47.297 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:58:47.503 4076 3051113163768 MainThread DEBUG nova.compute.provider_tree [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:58:47.538 4076 3051113163768 MainThread DEBUG nova.scheduler.client.report [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:58:47.538 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:47.538 4076 3051113163768 MainThread ERROR nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Traceback (most recent call last): 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(*args, **kwargs) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] admin_password, network_info, block_device_info) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(self, *args, **kwds) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] should_plug_vifs=False) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'req_state': req_state}) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] time.sleep(sleep_time) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return f(*args, **kwargs) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return self._wait_for_job(job_path) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] error_desc=err_desc) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] During handling of the above exception, another exception occurred: 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Traceback (most recent call last): 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] accel_info=accel_info) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(*args, **kwargs) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] admin_password, network_info, block_device_info) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return function(self, *args, **kwds) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] should_plug_vifs=False) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'req_state': req_state}) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] time.sleep(sleep_time) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self.force_reraise() 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] raise self.value 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return f(*args, **kwargs) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] return self._wait_for_job(job_path) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] error_desc=err_desc) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. 2021-09-10 16:58:47.538 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] 2021-09-10 16:58:47.538 4076 3051113163768 MainThread DEBUG nova.compute.utils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:58:47.538 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Build of instance 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003c' failed to start. Not enough memory in the system to start the virtual machine instance-0000003c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003c' failed to start. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Not enough memory in the system to start the virtual machine instance-0000003c with ram size 1024 megabytes. (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) 'instance-0000003c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 85FABA55-A2FD-4B6D-9800-A973AC530C2F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:58:47.538 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:58:47.554 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:58:47.554 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:58:47.554 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:58:47.554 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-b922a519-3efc-4024-abfb-de5c20d9d578 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:47.554 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-b922a519-3efc-4024-abfb-de5c20d9d578. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:47.554 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:47.554 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port b922a519-3efc-4024-abfb-de5c20d9d578 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:47.694 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:48.409 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:48.445 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8c7de3e4-6e54-484b-8c44-451bf6797d65 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:48.482 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:48.523 4076 3051113163768 MainThread INFO nova.compute.manager [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] [instance: 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9] Took 0.97 seconds to deallocate network for instance. 2021-09-10 16:58:48.810 4076 3051113163768 MainThread INFO nova.scheduler.client.report [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Deleted allocations for instance 04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9 2021-09-10 16:58:48.870 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-0fea709c-999a-4a82-886b-cd8493e0165b d62454119feb46319b4a349e02ddd6df a13940e712fe464c8a1b2c3732982edb - default default] Lock "04d0ca72-cd7c-4dce-a0e7-e59dcd94caa9" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.172s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:49.394 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:49.542 4076 3051113161304 MainThread INFO nova.compute.manager [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 1f6d373b-5624-411c-9545-511d98e94f86] Took 4.75 seconds to deallocate network for instance. 2021-09-10 16:58:49.854 4076 3051113161304 MainThread INFO nova.scheduler.client.report [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Deleted allocations for instance 1f6d373b-5624-411c-9545-511d98e94f86 2021-09-10 16:58:49.895 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-eb17a63f-9b92-413f-b335-2d1340f57f27 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "1f6d373b-5624-411c-9545-511d98e94f86" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:58:51.897 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-46dd9dca-3150-45cc-9a35-bc1230d1380e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:51.897 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-46dd9dca-3150-45cc-9a35-bc1230d1380e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:51.897 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:51.897 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port 46dd9dca-3150-45cc-9a35-bc1230d1380e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:52.350 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:52.709 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: 1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:58:53.748 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:53.769 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-b17dc05c-2a72-46ab-ac08-e45f70d4d258 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:55.041 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:58:55.041 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:58:55.041 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:58:55.041 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port 1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:58:55.551 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:58:57.106 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:58:57.126 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4ed61a5-505c-46eb-98c1-b431aa86399e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:58:57.823 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: 7210f512-68cf-4fad-b1ca-29f66e80f530 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:00.692 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-7210f512-68cf-4fad-b1ca-29f66e80f530 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:00.692 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-7210f512-68cf-4fad-b1ca-29f66e80f530. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:00.692 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:00.692 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port 7210f512-68cf-4fad-b1ca-29f66e80f530 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:01.085 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:01.242 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: bb905c29-b72e-4c4c-b524-e08c9df4bbd8 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:02.051 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:02.055 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-e63087ee-0950-402a-97a8-e62095b5152f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:04.192 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Successfully updated port: b083b498-e447-45f8-9086-ac65ceea526a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:04.207 4076 3051113162184 GreenThread-46 DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:04.207 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:04.564 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:05.616 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:05.631 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:05.756 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:05.756 4076 3051093169608 MainThread DEBUG nova.virt.hardware [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:05.756 4076 3051093169608 MainThread INFO nova.compute.claims [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Claim successful on node n-h1-808474-1 2021-09-10 16:59:05.770 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:05.770 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-bb905c29-b72e-4c4c-b524-e08c9df4bbd8. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:05.892 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:05.897 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:06.106 4076 3051093169608 MainThread DEBUG nova.compute.provider_tree [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:06.147 4076 3051093169608 MainThread DEBUG nova.scheduler.client.report [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:06.147 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:06.147 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:06.240 4076 3051093169608 MainThread DEBUG nova.compute.utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:06.254 4076 3051093170312 GreenThread-48 DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:06.254 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:06.278 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:06.447 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:06.464 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Spawning new instance 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG nova.block_device [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:06.464 4076 3051093169608 MainThread DEBUG os_win.utils.pathutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating directory: C:\openstack\instances\instance-0000003d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:06.490 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000003d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:06.493 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Resizing VHD C:\openstack\instances\instance-0000003d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:06.631 4076 3051093170312 GreenThread-48 DEBUG nova.policy [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '42267190de3648eea52604e6b2444d20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6ca07d9dc99a4581a698a232ffa065db', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:08.821 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Successfully created port: 60fba248-048a-479e-a90d-c8f335c2a060 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:09.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:09.518 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:09.600 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:09.615 4076 3051113162008 MainThread DEBUG nova.virt.hardware [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:09.615 4076 3051113162008 MainThread INFO nova.compute.claims [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Claim successful on node n-h1-808474-1 2021-09-10 16:59:09.718 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.hostops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:09.729 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.hostops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:09.873 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-changed-b083b498-e447-45f8-9086-ac65ceea526a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:09.873 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing instance network info cache due to event network-changed-b083b498-e447-45f8-9086-ac65ceea526a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:09.926 4076 3051113162008 MainThread DEBUG nova.compute.provider_tree [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:09.952 4076 3051113162008 MainThread DEBUG nova.scheduler.client.report [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:09.952 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:09.952 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:10.051 4076 3051113162008 MainThread DEBUG nova.compute.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:10.067 4076 3051113164120 GreenThread-49 DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:10.067 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:10.073 4076 3051113162008 MainThread DEBUG nova.compute.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:10.111 4076 3051113162008 MainThread DEBUG nova.compute.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:10.147 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:10.223 4076 3051113162008 MainThread INFO nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Booting with volume 36413d8d-e561-4803-ae3b-516b90e60d05 at /dev/sda 2021-09-10 16:59:10.301 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:10.301 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:10.301 4076 3051113162008 MainThread DEBUG nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating existing volume attachment record: ea95966f-29c3-4565-be14-afed273a0a11 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:59:10.323 4076 3051113164120 GreenThread-49 DEBUG nova.policy [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b736354cedde4200845a6c18d167eda3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fe5b53ea59bd43bf84431d6daec45a04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:10.991 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully created port: 9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:11.694 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Successfully updated port: 60fba248-048a-479e-a90d-c8f335c2a060 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:11.705 4076 3051093170312 GreenThread-48 DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Acquired lock "refresh_cache-44c45ac4-0d0a-4a8e-8398-7da035a852aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:11.705 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:11.867 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully created port: add97417-8a27-4260-8deb-932d14156785 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:12.176 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:12.925 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully created port: 1f3d3575-c85f-4f80-88aa-29e1aeca2b5a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:13.345 4076 3051093170312 GreenThread-48 DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Updating instance_info_cache with network_info: [{"id": "60fba248-048a-479e-a90d-c8f335c2a060", "address": "fa:16:3e:86:7c:73", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap60fba248-04", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:13.366 4076 3051093170312 GreenThread-48 DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Releasing lock "refresh_cache-44c45ac4-0d0a-4a8e-8398-7da035a852aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:13.366 4076 3051093170312 GreenThread-48 DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Instance network_info: |[{"id": "60fba248-048a-479e-a90d-c8f335c2a060", "address": "fa:16:3e:86:7c:73", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap60fba248-04", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:13.366 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Preparing to wait for external event network-vif-plugged-60fba248-048a-479e-a90d-c8f335c2a060 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:13.366 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:13.366 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:13.366 4076 3051093169608 MainThread DEBUG os_win.utils.compute.vmutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating VM instance-0000003d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:13.439 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Received event network-changed-60fba248-048a-479e-a90d-c8f335c2a060 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:13.439 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Refreshing instance network info cache due to event network-changed-60fba248-048a-479e-a90d-c8f335c2a060. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:13.439 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-44c45ac4-0d0a-4a8e-8398-7da035a852aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:13.439 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Refreshing network info cache for port 60fba248-048a-479e-a90d-c8f335c2a060 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:14.134 4076 3051113162184 GreenThread-46 DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [{"id": "4aebdc4f-3a01-4f71-aae2-147c0d08eeb1", "address": "fa:16:3e:2c:20:1b", "network": {"id": "0ea42bde-2b84-433b-b8cc-45cd597feea2", "bridge": null, "label": "tempest-TaggedBootDevicesTest-481914895-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4aebdc4f-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b922a519-3efc-4024-abfb-de5c20d9d578", "address": "fa:16:3e:52:ed:ea", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb922a519-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "46dd9dca-3150-45cc-9a35-bc1230d1380e", "address": "fa:16:3e:7d:39:9e", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap46dd9dca-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1", "address": "fa:16:3e:b6:68:d5", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.220", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1bdbd4e2-e2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7210f512-68cf-4fad-b1ca-29f66e80f530", "address": "fa:16:3e:bc:e3:ae", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7210f512-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bb905c29-b72e-4c4c-b524-e08c9df4bbd8", "address": "fa:16:3e:b3:1f:7f", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb905c29-b7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b083b498-e447-45f8-9086-ac65ceea526a", "address": "fa:16:3e:d0:1b:df", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb083b498-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:14.182 4076 3051113162184 GreenThread-46 DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:14.182 4076 3051113162184 GreenThread-46 DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance network_info: |[{"id": "4aebdc4f-3a01-4f71-aae2-147c0d08eeb1", "address": "fa:16:3e:2c:20:1b", "network": {"id": "0ea42bde-2b84-433b-b8cc-45cd597feea2", "bridge": null, "label": "tempest-TaggedBootDevicesTest-481914895-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4aebdc4f-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b922a519-3efc-4024-abfb-de5c20d9d578", "address": "fa:16:3e:52:ed:ea", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb922a519-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "46dd9dca-3150-45cc-9a35-bc1230d1380e", "address": "fa:16:3e:7d:39:9e", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap46dd9dca-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1", "address": "fa:16:3e:b6:68:d5", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.220", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1bdbd4e2-e2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7210f512-68cf-4fad-b1ca-29f66e80f530", "address": "fa:16:3e:bc:e3:ae", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7210f512-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bb905c29-b72e-4c4c-b524-e08c9df4bbd8", "address": "fa:16:3e:b3:1f:7f", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb905c29-b7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b083b498-e447-45f8-9086-ac65ceea526a", "address": "fa:16:3e:d0:1b:df", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb083b498-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:14.182 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:14.182 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port bb905c29-b72e-4c4c-b524-e08c9df4bbd8 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:14.182 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-b922a519-3efc-4024-abfb-de5c20d9d578 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-46dd9dca-3150-45cc-9a35-bc1230d1380e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-7210f512-68cf-4fad-b1ca-29f66e80f530 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Preparing to wait for external event network-vif-plugged-b083b498-e447-45f8-9086-ac65ceea526a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:14.204 4076 3051113161480 MainThread DEBUG os_win.utils.compute.vmutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Creating VM instance-0000003b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:14.258 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:14.496 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully created port: db6ca41c-54e9-4053-bd66-ca4521a281c9 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:14.665 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.volumeops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'target_lun': 0, 'volume_id': '29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'auth_method': 'CHAP', 'auth_username': 'rBnpxR9o56RqMYWmjN3D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'b0745679-4000-4aff-bd0a-29ded4711f24', 'attached_at': '', 'detached_at': '', 'volume_id': '29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'serial': '29c54fbe-e8c1-401b-a427-b0b007cae3a7'} to instance-0000003b _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:14.665 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'target_lun': 0, 'volume_id': '29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'auth_method': 'CHAP', 'auth_username': 'rBnpxR9o56RqMYWmjN3D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:14.665 4076 3051113161480 MainThread INFO os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:14.665 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:15.242 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully created port: f630db82-46c7-4de7-a3e1-4ede08bcb598 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:15.376 4076 3051113162008 MainThread INFO nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Booting with volume 174960b8-7aa2-43c8-aff0-31d5f7de9324 at /dev/sdb 2021-09-10 16:59:15.475 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:15.475 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:15.475 4076 3051113162008 MainThread DEBUG nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating existing volume attachment record: 93d262eb-c4ca-4937-802b-772cb75bc5d1 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:59:15.787 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Updated VIF entry in instance network info cache for port 60fba248-048a-479e-a90d-c8f335c2a060. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:15.787 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Updating instance_info_cache with network_info: [{"id": "60fba248-048a-479e-a90d-c8f335c2a060", "address": "fa:16:3e:86:7c:73", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap60fba248-04", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:15.819 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-2e36659a-7d89-4df1-90eb-63f5b20c58b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-44c45ac4-0d0a-4a8e-8398-7da035a852aa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:16.180 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: 9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:16.489 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updated VIF entry in instance network info cache for port bb905c29-b72e-4c4c-b524-e08c9df4bbd8. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:16.489 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [{"id": "4aebdc4f-3a01-4f71-aae2-147c0d08eeb1", "address": "fa:16:3e:2c:20:1b", "network": {"id": "0ea42bde-2b84-433b-b8cc-45cd597feea2", "bridge": null, "label": "tempest-TaggedBootDevicesTest-481914895-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4aebdc4f-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b922a519-3efc-4024-abfb-de5c20d9d578", "address": "fa:16:3e:52:ed:ea", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb922a519-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "46dd9dca-3150-45cc-9a35-bc1230d1380e", "address": "fa:16:3e:7d:39:9e", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap46dd9dca-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1", "address": "fa:16:3e:b6:68:d5", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.220", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1bdbd4e2-e2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7210f512-68cf-4fad-b1ca-29f66e80f530", "address": "fa:16:3e:bc:e3:ae", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7210f512-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bb905c29-b72e-4c4c-b524-e08c9df4bbd8", "address": "fa:16:3e:b3:1f:7f", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb905c29-b7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b083b498-e447-45f8-9086-ac65ceea526a", "address": "fa:16:3e:d0:1b:df", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb083b498-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:16.523 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-91718c38-a1a6-4f73-bc7e-a40c5cab3bb1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:16.523 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:16.523 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Refreshing network info cache for port b083b498-e447-45f8-9086-ac65ceea526a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:17.274 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: 2328bac5-09f9-4dd3-a3ff-1f5673edded4 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:17.747 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:17.747 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:17.747 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:17.747 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port 9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:18.490 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:18.542 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: ffc6e67e-0a49-4c79-9748-12da98323372 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:18.709 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-a5a18f75-44be-439e-840f-db2d2d6e0fad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Received event network-vif-plugged-60fba248-048a-479e-a90d-c8f335c2a060 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:18.709 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-a5a18f75-44be-439e-840f-db2d2d6e0fad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:18.716 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-a5a18f75-44be-439e-840f-db2d2d6e0fad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:18.716 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-a5a18f75-44be-439e-840f-db2d2d6e0fad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Processing event network-vif-plugged-60fba248-048a-479e-a90d-c8f335c2a060 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:18.749 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Using config drive for instance 2021-09-10 16:59:19.026 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Creating config drive at C:\openstack\instances\instance-0000003d\configdrive.iso 2021-09-10 16:59:19.082 4076 3051093169608 MainThread DEBUG oslo_concurrency.processutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa73xvzy1 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:59:19.226 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updated VIF entry in instance network info cache for port b083b498-e447-45f8-9086-ac65ceea526a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:19.226 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [{"id": "4aebdc4f-3a01-4f71-aae2-147c0d08eeb1", "address": "fa:16:3e:2c:20:1b", "network": {"id": "0ea42bde-2b84-433b-b8cc-45cd597feea2", "bridge": null, "label": "tempest-TaggedBootDevicesTest-481914895-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4aebdc4f-3a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b922a519-3efc-4024-abfb-de5c20d9d578", "address": "fa:16:3e:52:ed:ea", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.139", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb922a519-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "46dd9dca-3150-45cc-9a35-bc1230d1380e", "address": "fa:16:3e:7d:39:9e", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap46dd9dca-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1", "address": "fa:16:3e:b6:68:d5", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.220", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1bdbd4e2-e2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "7210f512-68cf-4fad-b1ca-29f66e80f530", "address": "fa:16:3e:bc:e3:ae", "network": {"id": "4ec0c006-6906-4edf-ab6c-82a39302b7c1", "bridge": null, "label": "tempest-device-tagging-net1-1870902891", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7210f512-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "bb905c29-b72e-4c4c-b524-e08c9df4bbd8", "address": "fa:16:3e:b3:1f:7f", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb905c29-b7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "b083b498-e447-45f8-9086-ac65ceea526a", "address": "fa:16:3e:d0:1b:df", "network": {"id": "4a8716e2-ae76-4874-9712-4b5d242ecfec", "bridge": null, "label": "tempest-device-tagging-net2-1694500291", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "f00087122c344aa8a15842d2b03f9a16", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb083b498-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:19.252 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-2e26d6b9-ab26-46bc-9912-dfd46e7cd9d8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0745679-4000-4aff-bd0a-29ded4711f24" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:19.411 4076 3051093169608 MainThread DEBUG oslo_concurrency.processutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa73xvzy1" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:59:19.462 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: add97417-8a27-4260-8deb-932d14156785 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:19.694 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:19.694 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a0478e6a-54f0-481f-b5ff-5e24ad573650 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:19.725 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:19.818 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Setting boot order for instance: instance-0000003d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:59:19.869 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:19.869 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:59:19.926 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'EBD14438-5777-402D-836F-63B90141FA3F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=26887), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:19.926 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== connect_volume: return (5261ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '3600140568B8D29B31BC405D925A4F7F6'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:20.038 4076 3051093170840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:20.059 4076 3051093170840 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:20.059 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:20.059 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:20.059 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EBD14438-5777-402D-836F-63B90141FA3F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117205), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:20.059 4076 3051093169608 MainThread DEBUG os_win._utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1898040771484375 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:20.059 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-dd471d62-673f-4c23-a7cc-4af4bed682a6 - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:20.059 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-dd471d62-673f-4c23-a7cc-4af4bed682a6 - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:20.129 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-dd471d62-673f-4c23-a7cc-4af4bed682a6 - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:20.177 4076 3051093170840 MainThread INFO nova.compute.manager [req-dd471d62-673f-4c23-a7cc-4af4bed682a6 - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:20.306 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-2328bac5-09f9-4dd3-a3ff-1f5673edded4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:20.306 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-2328bac5-09f9-4dd3-a3ff-1f5673edded4. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:20.306 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:20.306 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port 2328bac5-09f9-4dd3-a3ff-1f5673edded4 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:20.334 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.volumeops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'target_lun': 0, 'volume_id': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'auth_method': 'CHAP', 'auth_username': 'oi27LCgnLkajkpFVbsVr', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'b0745679-4000-4aff-bd0a-29ded4711f24', 'attached_at': '', 'detached_at': '', 'volume_id': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'serial': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84'} to instance-0000003b _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:20.334 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'target_lun': 0, 'volume_id': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'auth_method': 'CHAP', 'auth_username': 'oi27LCgnLkajkpFVbsVr', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:20.334 4076 3051113161480 MainThread INFO os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:20.348 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:20.362 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: 1f3d3575-c85f-4f80-88aa-29e1aeca2b5a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:20.547 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:21.084 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '033BA6B3-225E-477E-A6C1-6B015F338426', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18878), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:21.089 4076 3051113162008 MainThread INFO nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Booting with volume 7687f3e5-535f-43cb-92df-79afc5c86fb1 at /dev/sdc 2021-09-10 16:59:21.174 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:21.180 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:21.180 4076 3051113162008 MainThread DEBUG os_brick.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:21.180 4076 3051113162008 MainThread DEBUG nova.virt.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating existing volume attachment record: 1d820621-7408-48fb-acfa-3c0cfa4ebd83 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 16:59:21.195 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:21.195 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-ffc6e67e-0a49-4c79-9748-12da98323372 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:21.195 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-ffc6e67e-0a49-4c79-9748-12da98323372. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:21.195 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:21.195 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port ffc6e67e-0a49-4c79-9748-12da98323372 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:21.217 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:21.217 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:21.217 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:21.217 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:21.231 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-86397df2-2b6d-48ce-9c4b-df11203e9d0a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:21.242 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-86397df2-2b6d-48ce-9c4b-df11203e9d0a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:21.242 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '033BA6B3-225E-477E-A6C1-6B015F338426', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111006), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:21.242 4076 3051093169608 MainThread DEBUG os_win._utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.372959852218628 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:21.277 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: db6ca41c-54e9-4053-bd66-ca4521a281c9 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:21.288 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-86397df2-2b6d-48ce-9c4b-df11203e9d0a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:21.337 4076 3051093169784 MainThread INFO nova.compute.manager [req-86397df2-2b6d-48ce-9c4b-df11203e9d0a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:21.447 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:22.164 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Successfully updated port: f630db82-46c7-4de7-a3e1-4ede08bcb598 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:22.263 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'DC58893F-4A75-46EE-9DBB-C79CFED49A82', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14640), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:22.397 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:22.397 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:22.397 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:22.397 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:22.421 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-f626ead5-36a5-45ca-9314-2f09dcf84bae - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:22.421 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-f626ead5-36a5-45ca-9314-2f09dcf84bae - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:22.421 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DC58893F-4A75-46EE-9DBB-C79CFED49A82', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109381), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:22.421 4076 3051093169608 MainThread DEBUG os_win._utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5520575046539307 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:22.471 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-f626ead5-36a5-45ca-9314-2f09dcf84bae - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:22.522 4076 3051049643176 MainThread INFO nova.compute.manager [req-f626ead5-36a5-45ca-9314-2f09dcf84bae - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:22.772 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:22.810 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:22.810 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-add97417-8a27-4260-8deb-932d14156785 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:22.810 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-add97417-8a27-4260-8deb-932d14156785. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:22.810 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:22.810 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port add97417-8a27-4260-8deb-932d14156785 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:23.092 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:23.189 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:23.189 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:23.454 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'F3220E07-361F-40A1-A8D0-B4B3FE6B5A6B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21470), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:23.571 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:23.571 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:23.571 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:23.571 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:23.600 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F3220E07-361F-40A1-A8D0-B4B3FE6B5A6B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113966), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:23.600 4076 3051093169608 MainThread DEBUG os_win._utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.730550527572632 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:23.626 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-0bfa821f-621a-411a-adff-f2e2c428b22b - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:23.626 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-0bfa821f-621a-411a-adff-f2e2c428b22b - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:23.678 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-0bfa821f-621a-411a-adff-f2e2c428b22b - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:23.725 4076 3051093171016 MainThread INFO nova.compute.manager [req-0bfa821f-621a-411a-adff-f2e2c428b22b - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:23.771 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:23.805 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-791cac52-5bec-4462-8f1d-c95024d8e936 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:23.805 4076 3051113164120 GreenThread-49 DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:23.805 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:24.141 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:24.365 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:24.381 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:24.505 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:24.505 4076 3051093170840 MainThread DEBUG nova.virt.hardware [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:24.505 4076 3051093170840 MainThread INFO nova.compute.claims [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Claim successful on node n-h1-808474-1 2021-09-10 16:59:24.621 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '77DDC532-8EB8-491E-87BE-0E10593DCB39', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17364), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:24.621 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.hostops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:24.647 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.hostops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:24.758 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:24.758 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:24.758 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:24.774 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:24.774 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '77DDC532-8EB8-491E-87BE-0E10593DCB39', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112337), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:24.774 4076 3051093169608 MainThread DEBUG os_win._utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9047417640686035 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:24.789 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-ec6917b2-fc7d-4fa8-9872-4ffe338e376f - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:24.789 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-ec6917b2-fc7d-4fa8-9872-4ffe338e376f - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:24.821 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-ec6917b2-fc7d-4fa8-9872-4ffe338e376f - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:24.870 4076 3051093171016 MainThread INFO nova.compute.manager [req-ec6917b2-fc7d-4fa8-9872-4ffe338e376f - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:24.870 4076 3051093170840 MainThread DEBUG nova.compute.provider_tree [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:24.897 4076 3051093170840 MainThread DEBUG nova.scheduler.client.report [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:24.897 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:24.897 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:25.012 4076 3051093170840 MainThread DEBUG nova.compute.utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:25.012 4076 3051093171016 GreenThread-50 DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:25.012 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:25.041 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:25.209 4076 3051093171016 GreenThread-50 DEBUG nova.policy [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a1711a7b02c2433dacfb4017cd0a812c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b9dfa46b614e449b6aff88db2cfd6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:25.292 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:25.292 4076 3051093170840 MainThread INFO nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Spawning new instance 2021-09-10 16:59:25.292 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:25.292 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:25.292 4076 3051093170840 MainThread DEBUG nova.block_device [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:25.314 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:25.314 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:25.314 4076 3051093170840 MainThread DEBUG os_win.utils.pathutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating directory: C:\openstack\instances\instance-00000040 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:25.314 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000040\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:25.345 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Resizing VHD C:\openstack\instances\instance-00000040\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:25.381 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:25.476 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:25.507 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== connect_volume: return (5172ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive2', 'number': 2, 'scsi_wwn': '36001405A4DC49559F4E47F1A1A92856A'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:25.794 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '487B0C34-8D72-4EAB-BA39-4A8B4D9F56EE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15144), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:25.912 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.volumeops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8', 'target_lun': 0, 'volume_id': 'f0f70f22-4412-4d14-80e3-17ea362502a8', 'auth_method': 'CHAP', 'auth_username': 'V3gDNm77PePMLgud3BQ5', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'b0745679-4000-4aff-bd0a-29ded4711f24', 'attached_at': '', 'detached_at': '', 'volume_id': 'f0f70f22-4412-4d14-80e3-17ea362502a8', 'serial': 'f0f70f22-4412-4d14-80e3-17ea362502a8'} to instance-0000003b _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:25.912 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8', 'target_lun': 0, 'volume_id': 'f0f70f22-4412-4d14-80e3-17ea362502a8', 'auth_method': 'CHAP', 'auth_username': 'V3gDNm77PePMLgud3BQ5', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:25.912 4076 3051113161480 MainThread INFO os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:25.912 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:25.934 4076 3051113161656 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:25.934 4076 3051113161656 MainThread INFO nova.compute.manager [-] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] VM Stopped (Lifecycle Event) 2021-09-10 16:59:25.934 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:25.934 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:25.934 4076 3051093169608 MainThread ERROR os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003d\' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)instance-0000003d37F305EB-A960-453C-B876-1FAF5DF5D05B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '487B0C34-8D72-4EAB-BA39-4A8B4D9F56EE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114939), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\nNot enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)\n\n'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B)", 'ErrorSummaryDescription': "'instance-0000003d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:25.934 4076 3051093169608 MainThread ERROR nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Failed to change vm state of instance-0000003d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. 2021-09-10 16:59:25.948 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Got request to destroy instance 2021-09-10 16:59:25.960 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-a6b2171b-e026-4271-88f9-ec697bedf46a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:25.960 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-a6b2171b-e026-4271-88f9-ec697bedf46a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:26.010 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-a6b2171b-e026-4271-88f9-ec697bedf46a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:26.058 4076 3051113161656 MainThread INFO nova.compute.manager [req-a6b2171b-e026-4271-88f9-ec697bedf46a - - - - -] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:26.620 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Successfully created port: 4d2eb961-863f-4159-93af-5a4180653916 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:26.787 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:26.787 4076 3051113162008 MainThread INFO nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Spawning new instance 2021-09-10 16:59:26.787 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:26.787 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:26.787 4076 3051113162008 MainThread DEBUG nova.block_device [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] block_device_list ['sda', 'sdb', 'sdc'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:27.823 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Successfully updated port: 4d2eb961-863f-4159-93af-5a4180653916 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:27.850 4076 3051093171016 GreenThread-50 DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Acquired lock "refresh_cache-3b8ed741-f00d-4e2b-bac4-95e4f3380c89" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:27.850 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:27.850 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:59:27.850 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:27.850 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:27.867 4076 3051093169608 MainThread DEBUG os_win.utils.compute.vmutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed vm state of instance-0000003d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:59:27.867 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed state of VM instance-0000003d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:59:27.913 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '37EC6CEF-91F2-4D9B-B49B-6453E8F93915', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14723), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:27.950 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:28.038 4076 3051093169608 MainThread DEBUG os_win.utils.jobutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job succeeded: {'InstanceID': '37EC6CEF-91F2-4D9B-B49B-6453E8F93915', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28568), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:59:28.038 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:28.038 4076 3051093169608 MainThread DEBUG os_win.utils.pathutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Removing directory: C:\openstack\instances\instance-0000003d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:59:28.038 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:28.038 4076 3051093169608 MainThread ERROR nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Traceback (most recent call last): 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(*args, **kwargs) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] admin_password, network_info, block_device_info) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(self, *args, **kwds) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] should_plug_vifs=False) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'req_state': req_state}) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] time.sleep(sleep_time) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return f(*args, **kwargs) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return self._wait_for_job(job_path) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] error_desc=err_desc) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During handling of the above exception, another exception occurred: 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Traceback (most recent call last): 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] yield resources 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] accel_info=accel_info) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(*args, **kwargs) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] admin_password, network_info, block_device_info) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(self, *args, **kwds) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] should_plug_vifs=False) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'req_state': req_state}) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] time.sleep(sleep_time) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return f(*args, **kwargs) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return self._wait_for_job(job_path) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] error_desc=err_desc) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.038 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.038 4076 3051093169608 MainThread INFO nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Terminating instance 2021-09-10 16:59:28.038 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:59:28.038 4076 3051093169608 MainThread INFO nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Got request to destroy instance 2021-09-10 16:59:28.057 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.vmops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:59:28.057 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:28.057 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-0000003d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:28.057 4076 3051093169608 MainThread INFO nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:59:28.057 4076 3051093169608 MainThread DEBUG nova.compute.claims [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:59:28.057 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:28.209 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:28.219 4076 3051093169608 MainThread DEBUG nova.virt.hyperv.hostops [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:28.353 4076 3051093171016 GreenThread-50 DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Updating instance_info_cache with network_info: [{"id": "4d2eb961-863f-4159-93af-5a4180653916", "address": "fa:16:3e:e2:8c:7f", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4d2eb961-86", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:28.377 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Received event network-changed-4d2eb961-863f-4159-93af-5a4180653916 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:28.377 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Refreshing instance network info cache due to event network-changed-4d2eb961-863f-4159-93af-5a4180653916. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:28.396 4076 3051093171016 GreenThread-50 DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Releasing lock "refresh_cache-3b8ed741-f00d-4e2b-bac4-95e4f3380c89" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:28.396 4076 3051093171016 GreenThread-50 DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Instance network_info: |[{"id": "4d2eb961-863f-4159-93af-5a4180653916", "address": "fa:16:3e:e2:8c:7f", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4d2eb961-86", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:28.399 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Preparing to wait for external event network-vif-plugged-4d2eb961-863f-4159-93af-5a4180653916 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:28.399 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:28.399 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:28.399 4076 3051093170840 MainThread DEBUG os_win.utils.compute.vmutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating VM instance-00000040 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:28.399 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-3b8ed741-f00d-4e2b-bac4-95e4f3380c89" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:28.399 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Refreshing network info cache for port 4d2eb961-863f-4159-93af-5a4180653916 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:28.445 4076 3051093169608 MainThread DEBUG nova.compute.provider_tree [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:28.445 4076 3051093169608 MainThread DEBUG nova.scheduler.client.report [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:28.445 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:28.445 4076 3051093169608 MainThread ERROR nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Traceback (most recent call last): 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(*args, **kwargs) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] admin_password, network_info, block_device_info) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(self, *args, **kwds) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] should_plug_vifs=False) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'req_state': req_state}) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] time.sleep(sleep_time) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return f(*args, **kwargs) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return self._wait_for_job(job_path) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] error_desc=err_desc) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] During handling of the above exception, another exception occurred: 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Traceback (most recent call last): 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] accel_info=accel_info) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(*args, **kwargs) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] admin_password, network_info, block_device_info) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return function(self, *args, **kwds) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] should_plug_vifs=False) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'req_state': req_state}) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] time.sleep(sleep_time) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self.force_reraise() 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] raise self.value 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return f(*args, **kwargs) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] return self._wait_for_job(job_path) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] error_desc=err_desc) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. 2021-09-10 16:59:28.445 4076 3051093169608 MainThread TRACE nova.compute.manager [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.compute.utils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Build of instance 44c45ac4-0d0a-4a8e-8398-7da035a852aa was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003d' failed to start. Not enough memory in the system to start the virtual machine instance-0000003d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003d' failed to start. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Not enough memory in the system to start the virtual machine instance-0000003d with ram size 1024 megabytes. (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) 'instance-0000003d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 37F305EB-A960-453C-B876-1FAF5DF5D05B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:59:28.467 4076 3051093169608 MainThread DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:59:29.028 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Updated VIF entry in instance network info cache for port 4d2eb961-863f-4159-93af-5a4180653916. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:29.028 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Updating instance_info_cache with network_info: [{"id": "4d2eb961-863f-4159-93af-5a4180653916", "address": "fa:16:3e:e2:8c:7f", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4d2eb961-86", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:29.053 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-2ffe6906-61b2-4345-a793-cd33926b25be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-3b8ed741-f00d-4e2b-bac4-95e4f3380c89" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:29.337 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:29.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:30.005 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:30.599 4076 3051093169608 MainThread DEBUG nova.network.neutron [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:30.635 4076 3051093169608 MainThread INFO nova.compute.manager [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 44c45ac4-0d0a-4a8e-8398-7da035a852aa] Took 2.17 seconds to deallocate network for instance. 2021-09-10 16:59:30.912 4076 3051093169608 MainThread INFO nova.scheduler.client.report [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Deleted allocations for instance 44c45ac4-0d0a-4a8e-8398-7da035a852aa 2021-09-10 16:59:30.959 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:30.960 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-56081686-788f-483a-8081-2b43e77477aa 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "44c45ac4-0d0a-4a8e-8398-7da035a852aa" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.071 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:31.119 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== connect_volume: return (5207ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive3', 'number': 3, 'scsi_wwn': '36001405C689157BA7734366A00DEC9F9'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:31.415 4076 3051113164120 GreenThread-49 DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [{"id": "9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c", "address": "fa:16:3e:75:1e:2a", "network": {"id": "0970e3c9-902a-4302-863e-5a8822167335", "bridge": null, "label": "tempest-TaggedBootDevicesTest_v242-2094950958-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9f4f550b-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2328bac5-09f9-4dd3-a3ff-1f5673edded4", "address": "fa:16:3e:4f:71:a9", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2328bac5-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ffc6e67e-0a49-4c79-9748-12da98323372", "address": "fa:16:3e:18:10:c4", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.198", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapffc6e67e-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "add97417-8a27-4260-8deb-932d14156785", "address": "fa:16:3e:7d:63:cd", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.171", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapadd97417-8a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1f3d3575-c85f-4f80-88aa-29e1aeca2b5a", "address": "fa:16:3e:cd:04:8e", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.162", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1f3d3575-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "db6ca41c-54e9-4053-bd66-ca4521a281c9", "address": "fa:16:3e:70:c7:c2", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdb6ca41c-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f630db82-46c7-4de7-a3e1-4ede08bcb598", "address": "fa:16:3e:b8:a9:75", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf630db82-46", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:31.482 4076 3051113164120 GreenThread-49 DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:31.482 4076 3051113164120 GreenThread-49 DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance network_info: |[{"id": "9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c", "address": "fa:16:3e:75:1e:2a", "network": {"id": "0970e3c9-902a-4302-863e-5a8822167335", "bridge": null, "label": "tempest-TaggedBootDevicesTest_v242-2094950958-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9f4f550b-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2328bac5-09f9-4dd3-a3ff-1f5673edded4", "address": "fa:16:3e:4f:71:a9", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2328bac5-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ffc6e67e-0a49-4c79-9748-12da98323372", "address": "fa:16:3e:18:10:c4", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.198", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapffc6e67e-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "add97417-8a27-4260-8deb-932d14156785", "address": "fa:16:3e:7d:63:cd", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.171", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapadd97417-8a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1f3d3575-c85f-4f80-88aa-29e1aeca2b5a", "address": "fa:16:3e:cd:04:8e", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.162", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1f3d3575-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "db6ca41c-54e9-4053-bd66-ca4521a281c9", "address": "fa:16:3e:70:c7:c2", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdb6ca41c-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f630db82-46c7-4de7-a3e1-4ede08bcb598", "address": "fa:16:3e:b8:a9:75", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf630db82-46", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-ffc6e67e-0a49-4c79-9748-12da98323372 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.482 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-add97417-8a27-4260-8deb-932d14156785 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Preparing to wait for external event network-vif-plugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:31.493 4076 3051113162008 MainThread DEBUG os_win.utils.compute.vmutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Creating VM instance-0000003f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:31.493 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:31.493 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port 1f3d3575-c85f-4f80-88aa-29e1aeca2b5a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:31.596 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:31.666 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:31.744 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:31.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:31.803 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:31.803 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 16:59:31.819 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:31.917 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:32.011 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:32.024 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.volumeops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05', 'target_lun': 0, 'volume_id': '36413d8d-e561-4803-ae3b-516b90e60d05', 'auth_method': 'CHAP', 'auth_username': 'r7zZCaWyvbNG2TWUNX7m', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'd4b1aa5b-c1ae-4df7-afd7-15d31bbaa869', 'attached_at': '', 'detached_at': '', 'volume_id': '36413d8d-e561-4803-ae3b-516b90e60d05', 'serial': '36413d8d-e561-4803-ae3b-516b90e60d05'} to instance-0000003f _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:32.024 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05', 'target_lun': 0, 'volume_id': '36413d8d-e561-4803-ae3b-516b90e60d05', 'auth_method': 'CHAP', 'auth_username': 'r7zZCaWyvbNG2TWUNX7m', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:32.024 4076 3051113162008 MainThread INFO os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:32.024 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:32.085 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:32.631 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:32.631 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 16:59:34.178 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updated VIF entry in instance network info cache for port 1f3d3575-c85f-4f80-88aa-29e1aeca2b5a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:34.178 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [{"id": "9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c", "address": "fa:16:3e:75:1e:2a", "network": {"id": "0970e3c9-902a-4302-863e-5a8822167335", "bridge": null, "label": "tempest-TaggedBootDevicesTest_v242-2094950958-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9f4f550b-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2328bac5-09f9-4dd3-a3ff-1f5673edded4", "address": "fa:16:3e:4f:71:a9", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2328bac5-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ffc6e67e-0a49-4c79-9748-12da98323372", "address": "fa:16:3e:18:10:c4", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.198", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapffc6e67e-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "add97417-8a27-4260-8deb-932d14156785", "address": "fa:16:3e:7d:63:cd", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.171", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapadd97417-8a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1f3d3575-c85f-4f80-88aa-29e1aeca2b5a", "address": "fa:16:3e:cd:04:8e", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.162", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1f3d3575-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "db6ca41c-54e9-4053-bd66-ca4521a281c9", "address": "fa:16:3e:70:c7:c2", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdb6ca41c-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f630db82-46c7-4de7-a3e1-4ede08bcb598", "address": "fa:16:3e:b8:a9:75", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf630db82-46", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:34.209 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:34.209 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-db6ca41c-54e9-4053-bd66-ca4521a281c9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:34.209 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-db6ca41c-54e9-4053-bd66-ca4521a281c9. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:34.209 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:34.209 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port db6ca41c-54e9-4053-bd66-ca4521a281c9 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:35.770 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:35.787 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:35.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 16:59:35.802 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:35.802 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:35.818 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:36.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:36.802 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:36.829 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:36.829 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:36.829 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 16:59:36.829 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:36.849 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:37.047 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:37.047 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updated VIF entry in instance network info cache for port db6ca41c-54e9-4053-bd66-ca4521a281c9. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:37.047 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [{"id": "9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c", "address": "fa:16:3e:75:1e:2a", "network": {"id": "0970e3c9-902a-4302-863e-5a8822167335", "bridge": null, "label": "tempest-TaggedBootDevicesTest_v242-2094950958-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9f4f550b-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2328bac5-09f9-4dd3-a3ff-1f5673edded4", "address": "fa:16:3e:4f:71:a9", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2328bac5-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ffc6e67e-0a49-4c79-9748-12da98323372", "address": "fa:16:3e:18:10:c4", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.198", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapffc6e67e-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "add97417-8a27-4260-8deb-932d14156785", "address": "fa:16:3e:7d:63:cd", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.171", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapadd97417-8a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1f3d3575-c85f-4f80-88aa-29e1aeca2b5a", "address": "fa:16:3e:cd:04:8e", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.162", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1f3d3575-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "db6ca41c-54e9-4053-bd66-ca4521a281c9", "address": "fa:16:3e:70:c7:c2", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdb6ca41c-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f630db82-46c7-4de7-a3e1-4ede08bcb598", "address": "fa:16:3e:b8:a9:75", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf630db82-46", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:37.068 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:37.092 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-changed-f630db82-46c7-4de7-a3e1-4ede08bcb598 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:37.092 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing instance network info cache due to event network-changed-f630db82-46c7-4de7-a3e1-4ede08bcb598. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:37.092 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:37.092 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Refreshing network info cache for port f630db82-46c7-4de7-a3e1-4ede08bcb598 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:37.105 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2200MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 16:59:37.105 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:37.133 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:37.194 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== connect_volume: return (5170ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive4', 'number': 4, 'scsi_wwn': '36001405FAF2D1B3A6EB45AA8499B5364'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance b0745679-4000-4aff-bd0a-29ded4711f24 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.367 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.387 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 3b8ed741-f00d-4e2b-bac4-95e4f3380c89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 16:59:37.387 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 15 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 16:59:37.387 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=15872MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=15 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 16:59:37.433 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:37.444 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:37.668 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.volumeops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324', 'target_lun': 0, 'volume_id': '174960b8-7aa2-43c8-aff0-31d5f7de9324', 'auth_method': 'CHAP', 'auth_username': 'rYi4xVp8k8gT6zEJHGPP', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'd4b1aa5b-c1ae-4df7-afd7-15d31bbaa869', 'attached_at': '', 'detached_at': '', 'volume_id': '174960b8-7aa2-43c8-aff0-31d5f7de9324', 'serial': '174960b8-7aa2-43c8-aff0-31d5f7de9324'} to instance-0000003f _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:37.668 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324', 'target_lun': 0, 'volume_id': '174960b8-7aa2-43c8-aff0-31d5f7de9324', 'auth_method': 'CHAP', 'auth_username': 'rYi4xVp8k8gT6zEJHGPP', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:37.668 4076 3051113162008 MainThread INFO os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:37.679 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:37.694 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:37.716 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:37.716 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 16:59:37.716 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:38.634 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-29f6ff9a-064b-4479-b808-96d953bedd4d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Received event network-vif-plugged-4d2eb961-863f-4159-93af-5a4180653916 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:38.634 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-29f6ff9a-064b-4479-b808-96d953bedd4d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:38.634 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [req-29f6ff9a-064b-4479-b808-96d953bedd4d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:38.634 4076 3051093169608 MainThread DEBUG nova.compute.manager [req-29f6ff9a-064b-4479-b808-96d953bedd4d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Processing event network-vif-plugged-4d2eb961-863f-4159-93af-5a4180653916 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:38.677 4076 3051093170840 MainThread INFO nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Using config drive for instance 2021-09-10 16:59:39.652 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:39.677 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:39.780 4076 3051093170840 MainThread INFO nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Creating config drive at C:\openstack\instances\instance-00000040\configdrive.iso 2021-09-10 16:59:39.820 4076 3051093170840 MainThread DEBUG oslo_concurrency.processutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000040\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplglwbikb execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:59:39.990 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:39.990 4076 3051113293256 MainThread DEBUG nova.virt.hardware [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:39.990 4076 3051113293256 MainThread INFO nova.compute.claims [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Claim successful on node n-h1-808474-1 2021-09-10 16:59:40.136 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:40.146 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:40.167 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updated VIF entry in instance network info cache for port f630db82-46c7-4de7-a3e1-4ede08bcb598. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:40.167 4076 3051093169784 MainThread DEBUG nova.network.neutron [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [{"id": "9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c", "address": "fa:16:3e:75:1e:2a", "network": {"id": "0970e3c9-902a-4302-863e-5a8822167335", "bridge": null, "label": "tempest-TaggedBootDevicesTest_v242-2094950958-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9f4f550b-e4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2328bac5-09f9-4dd3-a3ff-1f5673edded4", "address": "fa:16:3e:4f:71:a9", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.237", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2328bac5-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "ffc6e67e-0a49-4c79-9748-12da98323372", "address": "fa:16:3e:18:10:c4", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.198", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapffc6e67e-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "add97417-8a27-4260-8deb-932d14156785", "address": "fa:16:3e:7d:63:cd", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.171", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapadd97417-8a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "1f3d3575-c85f-4f80-88aa-29e1aeca2b5a", "address": "fa:16:3e:cd:04:8e", "network": {"id": "92e4eab1-6925-43e4-9374-eabacfd179c1", "bridge": null, "label": "tempest-device-tagging-net1-2142628708", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.162", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap1f3d3575-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "db6ca41c-54e9-4053-bd66-ca4521a281c9", "address": "fa:16:3e:70:c7:c2", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdb6ca41c-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "f630db82-46c7-4de7-a3e1-4ede08bcb598", "address": "fa:16:3e:b8:a9:75", "network": {"id": "b6a4b24e-05b0-4e47-95a8-02f942067acb", "bridge": null, "label": "tempest-device-tagging-net2-1280552467", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.2"}}], "meta": {"injected": false, "tenant_id": "fe5b53ea59bd43bf84431d6daec45a04", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf630db82-46", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:40.167 4076 3051093170840 MainThread DEBUG oslo_concurrency.processutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000040\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplglwbikb" returned: 0 in 0.343s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:59:40.197 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-85f54f88-b044-4292-960c-deccf805938b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:40.383 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:40.413 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:40.413 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.421s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:40.413 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:40.527 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:40.527 4076 3051049643176 GreenThread-51 DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:40.527 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:40.614 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Setting boot order for instance: instance-00000040: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:59:40.652 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:59:40.695 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 16:59:40.695 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 16:59:40.711 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '2B08454F-802A-48C2-AD40-9B564171E9E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15694), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:40.727 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:40.755 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Spawning new instance 2021-09-10 16:59:40.762 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:40.762 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:40.762 4076 3051113293256 MainThread DEBUG nova.block_device [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:40.774 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:40.774 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:40.774 4076 3051113293256 MainThread DEBUG os_win.utils.pathutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Creating directory: C:\openstack\instances\instance-0000003e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:40.774 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000003e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:40.807 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Resizing VHD C:\openstack\instances\instance-0000003e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:40.872 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:40.872 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:40.872 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:40.872 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:40.872 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2B08454F-802A-48C2-AD40-9B564171E9E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=127474), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:40.872 4076 3051093170840 MainThread DEBUG os_win._utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.21928095817565918 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:40.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-9a8529a3-f67f-483f-983b-e73346667347 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:40.899 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-9a8529a3-f67f-483f-983b-e73346667347 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:40.961 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-9a8529a3-f67f-483f-983b-e73346667347 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:40.996 4076 3051049643176 GreenThread-51 DEBUG nova.policy [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0db4ede2f101460f9ab63db4d0d76a9a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '912aa6945f734abbbd12a58348a3bbbb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:40.996 4076 3051093170312 MainThread INFO nova.compute.manager [req-9a8529a3-f67f-483f-983b-e73346667347 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:41.101 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-11236e85-4b7d-41d8-ba9e-4a05a16f9bc8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Received event network-vif-deleted-889d1613-638f-45b3-8590-8ba5211a3c23 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:41.896 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'C704469E-6E77-470E-BDFB-B6DA94BA4F37', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21066), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:42.052 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:42.052 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:42.052 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:42.052 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:42.052 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C704469E-6E77-470E-BDFB-B6DA94BA4F37', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=123971), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:42.052 4076 3051093170840 MainThread DEBUG os_win._utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.400099277496338 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:42.083 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2a22a7d6-8cba-4bda-958c-9900e1c59201 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:42.083 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-2a22a7d6-8cba-4bda-958c-9900e1c59201 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:42.139 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-2a22a7d6-8cba-4bda-958c-9900e1c59201 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:42.183 4076 3051093169784 MainThread INFO nova.compute.manager [req-2a22a7d6-8cba-4bda-958c-9900e1c59201 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:42.703 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:42.824 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:42.834 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:42.867 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:42.921 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== connect_volume: return (5252ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive5', 'number': 5, 'scsi_wwn': '36001405FA0548741D104361903A09E54'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:43.027 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:43.027 4076 3051113292024 MainThread DEBUG nova.virt.hardware [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:43.027 4076 3051113292024 MainThread INFO nova.compute.claims [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Claim successful on node n-h1-808474-1 2021-09-10 16:59:43.087 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '1A1B2BC4-2B7B-48BF-8B59-6C1C6AAC8CDD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15347), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:43.166 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.hostops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:43.195 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.hostops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:43.226 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:43.226 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:43.226 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:43.226 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:43.243 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1A1B2BC4-2B7B-48BF-8B59-6C1C6AAC8CDD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116661), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:43.243 4076 3051093170840 MainThread DEBUG os_win._utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.590975284576416 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:43.243 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-b462fd7c-c74d-4f22-b29a-361529ae52ca - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:43.243 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-b462fd7c-c74d-4f22-b29a-361529ae52ca - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:43.310 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-b462fd7c-c74d-4f22-b29a-361529ae52ca - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:43.328 4076 3051093169784 MainThread INFO nova.compute.manager [req-b462fd7c-c74d-4f22-b29a-361529ae52ca - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:43.375 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.volumeops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1', 'target_lun': 0, 'volume_id': '7687f3e5-535f-43cb-92df-79afc5c86fb1', 'auth_method': 'CHAP', 'auth_username': 'zyJxKJaxT6B3MYRsZn7s', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'd4b1aa5b-c1ae-4df7-afd7-15d31bbaa869', 'attached_at': '', 'detached_at': '', 'volume_id': '7687f3e5-535f-43cb-92df-79afc5c86fb1', 'serial': '7687f3e5-535f-43cb-92df-79afc5c86fb1'} to instance-0000003f _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 16:59:43.375 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1', 'target_lun': 0, 'volume_id': '7687f3e5-535f-43cb-92df-79afc5c86fb1', 'auth_method': 'CHAP', 'auth_username': 'zyJxKJaxT6B3MYRsZn7s', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 16:59:43.375 4076 3051113162008 MainThread INFO os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 16:59:43.375 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 16:59:43.430 4076 3051113292024 MainThread DEBUG nova.compute.provider_tree [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:43.464 4076 3051113292024 MainThread DEBUG nova.scheduler.client.report [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:43.464 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:43.464 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:43.579 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Successfully created port: f437e79a-816b-489c-b7de-d5ddc4f1d5be _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:43.590 4076 3051113292024 MainThread DEBUG nova.compute.utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:43.600 4076 3051093169784 GreenThread-52 DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:43.600 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:43.631 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:43.854 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:43.854 4076 3051113292024 MainThread INFO nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Spawning new instance 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG nova.block_device [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG os_win.utils.pathutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Creating directory: C:\openstack\instances\instance-00000041 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:43.880 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000041\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:43.911 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Resizing VHD C:\openstack\instances\instance-00000041\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:43.974 4076 3051093169784 GreenThread-52 DEBUG nova.policy [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bab205b0b6fc4ec2b4ffc7f2fdf0a14e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '966c4d4813584979a85cc2a7055682b3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:44.285 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'F49AF4BB-1A76-486A-9799-FCA4E2CD48EC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14684), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:44.411 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:44.411 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:44.411 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:44.411 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:44.433 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-2a61ec01-56ad-42e8-90a4-da77ddf8ce68 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:44.433 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-2a61ec01-56ad-42e8-90a4-da77ddf8ce68 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:44.433 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F49AF4BB-1A76-486A-9799-FCA4E2CD48EC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110280), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:44.433 4076 3051093170840 MainThread DEBUG os_win._utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.780508279800415 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:44.468 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-2a61ec01-56ad-42e8-90a4-da77ddf8ce68 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:44.505 4076 3051113292728 MainThread INFO nova.compute.manager [req-2a61ec01-56ad-42e8-90a4-da77ddf8ce68 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:45.462 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '1C5FB8A3-326F-4ABD-9DC4-818D9616D09B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16542), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:45.618 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:45.618 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:45.618 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:45.618 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:45.618 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1C5FB8A3-326F-4ABD-9DC4-818D9616D09B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=128393), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:45.618 4076 3051093170840 MainThread DEBUG os_win._utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.965782642364502 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:45.632 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-59b182d0-53c4-4cab-9240-f038af0e6802 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:45.662 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-59b182d0-53c4-4cab-9240-f038af0e6802 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:45.701 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-59b182d0-53c4-4cab-9240-f038af0e6802 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:45.716 4076 3051113292728 MainThread INFO nova.compute.manager [req-59b182d0-53c4-4cab-9240-f038af0e6802 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:45.900 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Successfully updated port: f437e79a-816b-489c-b7de-d5ddc4f1d5be _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:45.900 4076 3051049643176 GreenThread-51 DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Acquired lock "refresh_cache-aa775114-9ae9-476f-b4ed-2b6e1846f3bc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:45.900 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:46.157 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:46.349 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:46.364 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:46.474 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Successfully created port: 88725264-1291-47ff-9cc0-13502b9a8801 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:46.533 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:46.552 4076 3051113293784 MainThread DEBUG nova.virt.hardware [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:46.552 4076 3051113293784 MainThread INFO nova.compute.claims [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Claim successful on node n-h1-808474-1 2021-09-10 16:59:46.662 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'FECAC7B2-980F-4467-ADAE-E349387C3AFB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=31395), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:46.712 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.hostops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:46.712 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.hostops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:46.757 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-62ae1e0b-2491-44cf-b471-c533f2b0d738 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:46.757 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-62ae1e0b-2491-44cf-b471-c533f2b0d738 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:46.757 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-62ae1e0b-2491-44cf-b471-c533f2b0d738 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:46.757 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-62ae1e0b-2491-44cf-b471-c533f2b0d738 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:46.788 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:46.788 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:46.788 4076 3051113291848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:46.788 4076 3051113291848 MainThread INFO nova.compute.manager [-] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] VM Stopped (Lifecycle Event) 2021-09-10 16:59:46.788 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-2cc261b3-979e-470c-9e34-855c62c03bb8 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:46.818 4076 3051113291848 MainThread DEBUG nova.virt.hyperv.vmops [req-2cc261b3-979e-470c-9e34-855c62c03bb8 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:46.821 4076 3051093170840 MainThread ERROR os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000040\' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A17210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000040\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)instance-00000040FAF2B206-F413-4CE1-BF0C-FC2463F3A172%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FECAC7B2-980F-4467-ADAE-E349387C3AFB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114850), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\nNot enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)\n\n'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172)", 'ErrorSummaryDescription': "'instance-00000040' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000040.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:46.821 4076 3051093170840 MainThread ERROR nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Failed to change vm state of instance-00000040 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. 2021-09-10 16:59:46.821 4076 3051093170840 MainThread INFO nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Got request to destroy instance 2021-09-10 16:59:46.872 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-2cc261b3-979e-470c-9e34-855c62c03bb8 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:46.900 4076 3051113291848 MainThread INFO nova.compute.manager [req-2cc261b3-979e-470c-9e34-855c62c03bb8 - - - - -] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:46.979 4076 3051113293784 MainThread DEBUG nova.compute.provider_tree [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:47.003 4076 3051113293784 MainThread DEBUG nova.scheduler.client.report [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:47.013 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:47.013 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:47.085 4076 3051113293784 MainThread DEBUG nova.compute.utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:47.085 4076 3051113291848 GreenThread-53 DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:47.085 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:47.102 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:47.149 4076 3051049643176 GreenThread-51 DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Updating instance_info_cache with network_info: [{"id": "f437e79a-816b-489c-b7de-d5ddc4f1d5be", "address": "fa:16:3e:13:83:c3", "network": {"id": "9cad4abd-c7da-438a-bf2a-8b127ecf1997", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-1013471033-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "912aa6945f734abbbd12a58348a3bbbb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf437e79a-81", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:47.149 4076 3051049643176 GreenThread-51 DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Releasing lock "refresh_cache-aa775114-9ae9-476f-b4ed-2b6e1846f3bc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:47.149 4076 3051049643176 GreenThread-51 DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Instance network_info: |[{"id": "f437e79a-816b-489c-b7de-d5ddc4f1d5be", "address": "fa:16:3e:13:83:c3", "network": {"id": "9cad4abd-c7da-438a-bf2a-8b127ecf1997", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-1013471033-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "912aa6945f734abbbd12a58348a3bbbb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf437e79a-81", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:47.177 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Preparing to wait for external event network-vif-plugged-f437e79a-816b-489c-b7de-d5ddc4f1d5be prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:47.177 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:47.177 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:47.177 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Creating VM instance-0000003e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:47.216 4076 3051113291848 GreenThread-53 DEBUG nova.policy [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:47.322 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:47.338 4076 3051113293784 MainThread INFO nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Spawning new instance 2021-09-10 16:59:47.338 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG nova.block_device [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG os_win.utils.pathutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-00000042 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:47.350 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000042\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:47.385 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Resizing VHD C:\openstack\instances\instance-00000042\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:47.868 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Received event network-changed-f437e79a-816b-489c-b7de-d5ddc4f1d5be external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:47.868 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Refreshing instance network info cache due to event network-changed-f437e79a-816b-489c-b7de-d5ddc4f1d5be. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:47.881 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-aa775114-9ae9-476f-b4ed-2b6e1846f3bc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:47.881 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Refreshing network info cache for port f437e79a-816b-489c-b7de-d5ddc4f1d5be _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:47.896 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Successfully created port: e54ef2bd-544c-413d-8411-9b83ad7fc56c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:48.226 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:48.396 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 16:59:48.413 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Successfully updated port: 88725264-1291-47ff-9cc0-13502b9a8801 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:48.429 4076 3051093169784 GreenThread-52 DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Acquired lock "refresh_cache-5b7f9971-39c8-4512-b873-8d2e09fd6941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:48.429 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:48.493 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:48.513 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 16:59:48.524 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 16:59:48.557 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== connect_volume: return (5182ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive6', 'number': 6, 'scsi_wwn': '360014059A7755D643544067A408684B1'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 16:59:48.631 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:48.631 4076 3051113294840 MainThread DEBUG nova.virt.hardware [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 16:59:48.631 4076 3051113294840 MainThread INFO nova.compute.claims [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Claim successful on node n-h1-808474-1 2021-09-10 16:59:48.680 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Successfully updated port: e54ef2bd-544c-413d-8411-9b83ad7fc56c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:48.693 4076 3051113291848 GreenThread-53 DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-b1382ef7-107d-4113-a901-1877f9724cd1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:48.693 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:48.789 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:48.789 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:48.789 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:48.994 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:49.058 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:49.086 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:49.086 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.086 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 16:59:49.146 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.146 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 16:59:49.146 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.146 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.146 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 16:59:49.170 4076 3051113294664 GreenThread-54 DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 16:59:49.170 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 16:59:49.181 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 16:59:49.181 4076 3051093170840 MainThread DEBUG os_win.utils.compute.vmutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed vm state of instance-00000040 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 16:59:49.181 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed state of VM instance-00000040 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 16:59:49.229 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.248 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'CD4FF8A6-AE16-406D-89D2-69A9B7A1CFC4', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16364), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:49.314 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.358 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 16:59:49.370 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Spawning new instance 2021-09-10 16:59:49.370 4076 3051093170840 MainThread DEBUG os_win.utils.jobutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job succeeded: {'InstanceID': 'CD4FF8A6-AE16-406D-89D2-69A9B7A1CFC4', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32904), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 16:59:49.370 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.370 4076 3051093170840 MainThread DEBUG os_win.utils.pathutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Removing directory: C:\openstack\instances\instance-00000040 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 16:59:49.381 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.381 4076 3051093170840 MainThread ERROR nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Traceback (most recent call last): 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(*args, **kwargs) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] admin_password, network_info, block_device_info) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(self, *args, **kwds) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] should_plug_vifs=False) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'req_state': req_state}) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] time.sleep(sleep_time) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return f(*args, **kwargs) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return self._wait_for_job(job_path) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] error_desc=err_desc) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During handling of the above exception, another exception occurred: 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Traceback (most recent call last): 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] yield resources 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] accel_info=accel_info) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(*args, **kwargs) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] admin_password, network_info, block_device_info) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(self, *args, **kwds) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] should_plug_vifs=False) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'req_state': req_state}) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] time.sleep(sleep_time) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return f(*args, **kwargs) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return self._wait_for_job(job_path) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] error_desc=err_desc) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.381 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.381 4076 3051093170840 MainThread INFO nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Terminating instance 2021-09-10 16:59:49.381 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 16:59:49.381 4076 3051093170840 MainThread INFO nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Got request to destroy instance 2021-09-10 16:59:49.381 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.381 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.381 4076 3051113294840 MainThread DEBUG nova.block_device [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 16:59:49.381 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.381 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.381 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.400 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Creating directory: C:\openstack\instances\instance-00000043 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 16:59:49.400 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000043\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 16:59:49.400 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 16:59:49.400 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.400 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000040" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.400 4076 3051093170840 MainThread INFO nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 16:59:49.400 4076 3051093170840 MainThread DEBUG nova.compute.claims [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 16:59:49.400 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.436 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Resizing VHD C:\openstack\instances\instance-00000043\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 16:59:49.461 4076 3051113294664 GreenThread-54 DEBUG nova.policy [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '07d38d374c614cedbb4fed9d7dc73f45', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b1e8ca5b70b4088b2d52a886b333d20', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 16:59:49.461 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.540 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.hostops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 16:59:49.557 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.569 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.hostops [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 16:59:49.640 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:49.696 4076 3051113291848 GreenThread-53 DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Updating instance_info_cache with network_info: [{"id": "e54ef2bd-544c-413d-8411-9b83ad7fc56c", "address": "fa:16:3e:f8:d3:51", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape54ef2bd-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:49.725 4076 3051113291848 GreenThread-53 DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-b1382ef7-107d-4113-a901-1877f9724cd1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:49.725 4076 3051113291848 GreenThread-53 DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Instance network_info: |[{"id": "e54ef2bd-544c-413d-8411-9b83ad7fc56c", "address": "fa:16:3e:f8:d3:51", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape54ef2bd-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:49.742 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Preparing to wait for external event network-vif-plugged-e54ef2bd-544c-413d-8411-9b83ad7fc56c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:49.742 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.742 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.742 4076 3051113293784 MainThread DEBUG os_win.utils.compute.vmutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-00000042 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:49.794 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Received event network-changed-e54ef2bd-544c-413d-8411-9b83ad7fc56c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.794 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Refreshing instance network info cache due to event network-changed-e54ef2bd-544c-413d-8411-9b83ad7fc56c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:49.794 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b1382ef7-107d-4113-a901-1877f9724cd1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:49.794 4076 3051049643176 MainThread DEBUG nova.network.neutron [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Refreshing network info cache for port e54ef2bd-544c-413d-8411-9b83ad7fc56c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:49.823 4076 3051093170840 MainThread DEBUG nova.compute.provider_tree [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 16:59:49.842 4076 3051093170840 MainThread DEBUG nova.scheduler.client.report [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 16:59:49.842 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.842 4076 3051093170840 MainThread ERROR nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Traceback (most recent call last): 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(*args, **kwargs) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] admin_password, network_info, block_device_info) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(self, *args, **kwds) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] should_plug_vifs=False) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'req_state': req_state}) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] time.sleep(sleep_time) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return f(*args, **kwargs) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return self._wait_for_job(job_path) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] error_desc=err_desc) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] During handling of the above exception, another exception occurred: 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Traceback (most recent call last): 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] accel_info=accel_info) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise exc.with_traceback(exc_info[2]) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(*args, **kwargs) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] admin_password, network_info, block_device_info) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return function(self, *args, **kwds) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.destroy(instance, network_info, block_device_info) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] should_plug_vifs=False) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'req_state': req_state}) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] time.sleep(sleep_time) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self.force_reraise() 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] raise self.value 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return f(*args, **kwargs) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] return self._wait_for_job(job_path) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] error_desc=err_desc) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. 2021-09-10 16:59:49.842 4076 3051093170840 MainThread TRACE nova.compute.manager [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.compute.utils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Build of instance 3b8ed741-f00d-4e2b-bac4-95e4f3380c89 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000040' failed to start. Not enough memory in the system to start the virtual machine instance-00000040. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000040' failed to start. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Not enough memory in the system to start the virtual machine instance-00000040 with ram size 1024 megabytes. (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) 'instance-00000040' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FAF2B206-F413-4CE1-BF0C-FC2463F3A172) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 16:59:49.851 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 16:59:49.869 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Updated VIF entry in instance network info cache for port f437e79a-816b-489c-b7de-d5ddc4f1d5be. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:49.869 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Updating instance_info_cache with network_info: [{"id": "f437e79a-816b-489c-b7de-d5ddc4f1d5be", "address": "fa:16:3e:13:83:c3", "network": {"id": "9cad4abd-c7da-438a-bf2a-8b127ecf1997", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-1013471033-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "912aa6945f734abbbd12a58348a3bbbb", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf437e79a-81", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-71e4bcf8-e744-4aee-9abe-3dff5b89c4f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-aa775114-9ae9-476f-b4ed-2b6e1846f3bc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-46dd9dca-3150-45cc-9a35-bc1230d1380e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-46dd9dca-3150-45cc-9a35-bc1230d1380e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-b083b498-e447-45f8-9086-ac65ceea526a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-b083b498-e447-45f8-9086-ac65ceea526a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.899 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-b922a519-3efc-4024-abfb-de5c20d9d578 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-b922a519-3efc-4024-abfb-de5c20d9d578 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-7210f512-68cf-4fad-b1ca-29f66e80f530 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-7210f512-68cf-4fad-b1ca-29f66e80f530 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-plugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:49.918 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-83ac24b1-e3de-4b32-acfd-df662c8d93f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Processing event network-vif-plugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 16:59:49.982 4076 3051113161480 MainThread INFO nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Using config drive for instance 2021-09-10 16:59:50.224 4076 3051093169784 GreenThread-52 DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Updating instance_info_cache with network_info: [{"id": "88725264-1291-47ff-9cc0-13502b9a8801", "address": "fa:16:3e:1b:68:91", "network": {"id": "96edf7ac-f423-4067-abd8-1b3f2098a058", "bridge": null, "label": "tempest-ServerActionsTestJSON-421244876-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "966c4d4813584979a85cc2a7055682b3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap88725264-12", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:50.244 4076 3051093169784 GreenThread-52 DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Releasing lock "refresh_cache-5b7f9971-39c8-4512-b873-8d2e09fd6941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:50.244 4076 3051093169784 GreenThread-52 DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Instance network_info: |[{"id": "88725264-1291-47ff-9cc0-13502b9a8801", "address": "fa:16:3e:1b:68:91", "network": {"id": "96edf7ac-f423-4067-abd8-1b3f2098a058", "bridge": null, "label": "tempest-ServerActionsTestJSON-421244876-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "966c4d4813584979a85cc2a7055682b3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap88725264-12", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:50.244 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Preparing to wait for external event network-vif-plugged-88725264-1291-47ff-9cc0-13502b9a8801 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:50.244 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:50.244 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:50.266 4076 3051113292024 MainThread DEBUG os_win.utils.compute.vmutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Creating VM instance-00000041 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:50.351 4076 3051113161480 MainThread INFO nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Creating config drive at C:\openstack\instances\instance-0000003b\configdrive.iso 2021-09-10 16:59:50.400 4076 3051113161480 MainThread DEBUG oslo_concurrency.processutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmppf0hmm26 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 16:59:50.570 4076 3051112927656 MainThread DEBUG nova.compute.manager [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Received event network-changed-88725264-1291-47ff-9cc0-13502b9a8801 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:50.570 4076 3051112927656 MainThread DEBUG nova.compute.manager [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Refreshing instance network info cache due to event network-changed-88725264-1291-47ff-9cc0-13502b9a8801. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:50.570 4076 3051112927656 MainThread DEBUG oslo_concurrency.lockutils [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-5b7f9971-39c8-4512-b873-8d2e09fd6941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:50.570 4076 3051112927656 MainThread DEBUG nova.network.neutron [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Refreshing network info cache for port 88725264-1291-47ff-9cc0-13502b9a8801 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:50.592 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Successfully created port: 14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 16:59:50.774 4076 3051113161480 MainThread DEBUG oslo_concurrency.processutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmppf0hmm26" returned: 0 in 0.375s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 16:59:50.974 4076 3051049643176 MainThread DEBUG nova.network.neutron [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Updated VIF entry in instance network info cache for port e54ef2bd-544c-413d-8411-9b83ad7fc56c. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:50.974 4076 3051049643176 MainThread DEBUG nova.network.neutron [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Updating instance_info_cache with network_info: [{"id": "e54ef2bd-544c-413d-8411-9b83ad7fc56c", "address": "fa:16:3e:f8:d3:51", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape54ef2bd-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:51.000 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-83e9095b-953b-4e31-8aad-e3a545009c57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b1382ef7-107d-4113-a901-1877f9724cd1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:51.026 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:51.244 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Setting boot order for instance: instance-0000003b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 16:59:51.302 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 16:59:51.302 4076 3051093170840 MainThread DEBUG nova.network.neutron [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:51.318 4076 3051093170840 MainThread INFO nova.compute.manager [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: 3b8ed741-f00d-4e2b-bac4-95e4f3380c89] Took 1.47 seconds to deallocate network for instance. 2021-09-10 16:59:51.334 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': '4191DC7F-0CF2-42B7-AC57-3D0608E33E6C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15749), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:51.432 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:51.453 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Successfully updated port: 14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 16:59:51.464 4076 3051113294664 GreenThread-54 DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Acquired lock "refresh_cache-0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:51.464 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 16:59:51.562 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 16:59:51.589 4076 3051093170840 MainThread INFO nova.scheduler.client.report [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Deleted allocations for instance 3b8ed741-f00d-4e2b-bac4-95e4f3380c89 2021-09-10 16:59:51.656 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [req-9cb72150-302b-4045-8b23-52854b8d14b5 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "3b8ed741-f00d-4e2b-bac4-95e4f3380c89" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.296s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:51.712 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4191DC7F-0CF2-42B7-AC57-3D0608E33E6C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=323957), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:51.728 4076 3051113161480 MainThread DEBUG os_win._utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.4256408214569092 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:51.728 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:51.728 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:51.728 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:51.728 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:51.789 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-7d2f0b95-524e-4f10-ba34-d5e966ffcf99 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:51.789 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-7d2f0b95-524e-4f10-ba34-d5e966ffcf99 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:51.838 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-7d2f0b95-524e-4f10-ba34-d5e966ffcf99 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:51.896 4076 3051113293960 MainThread INFO nova.compute.manager [req-7d2f0b95-524e-4f10-ba34-d5e966ffcf99 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:51.943 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Received event network-changed-14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:51.943 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Refreshing instance network info cache due to event network-changed-14fcfe75-bd5b-4648-81e1-e5b0a8862bd7. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 16:59:52.029 4076 3051112927656 MainThread DEBUG nova.network.neutron [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Updated VIF entry in instance network info cache for port 88725264-1291-47ff-9cc0-13502b9a8801. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:52.052 4076 3051112927656 MainThread DEBUG nova.network.neutron [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Updating instance_info_cache with network_info: [{"id": "88725264-1291-47ff-9cc0-13502b9a8801", "address": "fa:16:3e:1b:68:91", "network": {"id": "96edf7ac-f423-4067-abd8-1b3f2098a058", "bridge": null, "label": "tempest-ServerActionsTestJSON-421244876-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "966c4d4813584979a85cc2a7055682b3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap88725264-12", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:52.052 4076 3051113294664 GreenThread-54 DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Updating instance_info_cache with network_info: [{"id": "14fcfe75-bd5b-4648-81e1-e5b0a8862bd7", "address": "fa:16:3e:b8:29:5e", "network": {"id": "3a45771a-2941-4756-b943-dca54c4cf7be", "bridge": null, "label": "tempest-InstanceActionsV221TestJSON-397585230-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4b1e8ca5b70b4088b2d52a886b333d20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap14fcfe75-bd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:52.067 4076 3051112927656 MainThread DEBUG oslo_concurrency.lockutils [req-9782c1b1-4d44-43f2-a50e-07de81c1cc68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-5b7f9971-39c8-4512-b873-8d2e09fd6941" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:52.099 4076 3051113294664 GreenThread-54 DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Releasing lock "refresh_cache-0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:52.099 4076 3051113294664 GreenThread-54 DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Instance network_info: |[{"id": "14fcfe75-bd5b-4648-81e1-e5b0a8862bd7", "address": "fa:16:3e:b8:29:5e", "network": {"id": "3a45771a-2941-4756-b943-dca54c4cf7be", "bridge": null, "label": "tempest-InstanceActionsV221TestJSON-397585230-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4b1e8ca5b70b4088b2d52a886b333d20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap14fcfe75-bd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 16:59:52.099 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 16:59:52.099 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Refreshing network info cache for port 14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 16:59:52.099 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Preparing to wait for external event network-vif-plugged-14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 16:59:52.099 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:52.099 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:52.099 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Creating VM instance-00000043 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 16:59:52.714 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Updated VIF entry in instance network info cache for port 14fcfe75-bd5b-4648-81e1-e5b0a8862bd7. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 16:59:52.714 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Updating instance_info_cache with network_info: [{"id": "14fcfe75-bd5b-4648-81e1-e5b0a8862bd7", "address": "fa:16:3e:b8:29:5e", "network": {"id": "3a45771a-2941-4756-b943-dca54c4cf7be", "bridge": null, "label": "tempest-InstanceActionsV221TestJSON-397585230-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "4b1e8ca5b70b4088b2d52a886b333d20", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap14fcfe75-bd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 16:59:52.731 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-482178c4-7598-413a-bd87-f4529eb477d4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 16:59:52.761 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': '1BB78706-15A0-4DE7-A87E-602B3F291A8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18308), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:53.163 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 16:59:53.163 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1BB78706-15A0-4DE7-A87E-602B3F291A8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=347039), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:53.163 4076 3051113161480 MainThread DEBUG os_win._utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.8605504035949707 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:53.196 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:53.196 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:53.196 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:53.196 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:53.215 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-153af2a4-7f8b-414c-8998-e8dea84b108d - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:53.215 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-153af2a4-7f8b-414c-8998-e8dea84b108d - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:53.272 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-153af2a4-7f8b-414c-8998-e8dea84b108d - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:53.302 4076 3051049643176 MainThread INFO nova.compute.manager [req-153af2a4-7f8b-414c-8998-e8dea84b108d - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:54.212 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': '3B7DF3C7-23D6-42EC-8E49-D8EC51E40EEA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23302), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:54.586 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3B7DF3C7-23D6-42EC-8E49-D8EC51E40EEA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=333074), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:54.586 4076 3051113161480 MainThread DEBUG os_win._utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 3.2838327884674072 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:54.608 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:54.608 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:54.608 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:54.621 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:54.631 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-37958946-64b4-486b-801a-c41906631902 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:54.631 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-37958946-64b4-486b-801a-c41906631902 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:54.694 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-37958946-64b4-486b-801a-c41906631902 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:54.718 4076 3051049643176 MainThread INFO nova.compute.manager [req-37958946-64b4-486b-801a-c41906631902 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:55.636 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': 'A44DD7D5-E1F9-442A-B074-6337FD855EC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=25369), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:56.042 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:56.042 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:56.042 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:56.042 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:56.053 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A44DD7D5-E1F9-442A-B074-6337FD855EC3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=330291), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:56.053 4076 3051113161480 MainThread DEBUG os_win._utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.750262498855591 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:56.053 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-998e8b47-a462-44b7-891a-efb821313177 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:56.053 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-998e8b47-a462-44b7-891a-efb821313177 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:56.103 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-998e8b47-a462-44b7-891a-efb821313177 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:56.131 4076 3051049643176 MainThread INFO nova.compute.manager [req-998e8b47-a462-44b7-891a-efb821313177 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:57.092 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': '130290A8-964A-416E-811E-78E685D33253', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15942), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:57.524 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:57.524 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:57.524 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:57.524 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:57.524 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '130290A8-964A-416E-811E-78E685D33253', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=355220), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:57.524 4076 3051113161480 MainThread DEBUG os_win._utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 6.221430540084839 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 16:59:57.539 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-dbf7a79b-b281-4dbf-91fd-eab7ce647fb4 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:57.539 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-dbf7a79b-b281-4dbf-91fd-eab7ce647fb4 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:57.605 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-dbf7a79b-b281-4dbf-91fd-eab7ce647fb4 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:57.620 4076 3051049643176 MainThread INFO nova.compute.manager [req-dbf7a79b-b281-4dbf-91fd-eab7ce647fb4 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:58.570 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': '6736B470-9F41-4138-8126-6B23993D46D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14926), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 16:59:58.947 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:58.947 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:58.947 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 16:59:58.959 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] VM Stopped (Lifecycle Event) 2021-09-10 16:59:58.959 4076 3051113161480 MainThread ERROR os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003b\' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)instance-0000003b4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6736B470-9F41-4138-8126-6B23993D46D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=309583), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 16, 59, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 16, 59, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\nNot enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)\n\n'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8)", 'ErrorSummaryDescription': "'instance-0000003b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 16:59:58.959 4076 3051113161480 MainThread ERROR nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Failed to change vm state of instance-0000003b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. 2021-09-10 16:59:58.959 4076 3051113161480 MainThread INFO nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Got request to destroy instance 2021-09-10 16:59:58.959 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-a516db66-1725-4728-8d61-487e4acba0a8 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 16:59:58.959 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-a516db66-1725-4728-8d61-487e4acba0a8 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 16:59:59.038 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-a516db66-1725-4728-8d61-487e4acba0a8 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 16:59:59.068 4076 3051049643176 MainThread INFO nova.compute.manager [req-a516db66-1725-4728-8d61-487e4acba0a8 - - - - -] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 16:59:59.443 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-2efec281-719c-453b-bdd4-f50a362dbf7c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 16:59:59.443 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-2efec281-719c-453b-bdd4-f50a362dbf7c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 16:59:59.443 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-2efec281-719c-453b-bdd4-f50a362dbf7c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 16:59:59.443 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-2efec281-719c-453b-bdd4-f50a362dbf7c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Received event network-vif-plugged-f437e79a-816b-489c-b7de-d5ddc4f1d5be external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:01.693 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-e75922e1-c318-42a8-87f4-9637103692cd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Processing event network-vif-plugged-f437e79a-816b-489c-b7de-d5ddc4f1d5be _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:01.736 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Using config drive for instance 2021-09-10 17:00:01.837 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Creating config drive at C:\openstack\instances\instance-0000003e\configdrive.iso 2021-09-10 17:00:01.890 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpgdyu0nr1 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:02.177 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:02.177 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:02.177 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:02.193 4076 3051113161480 MainThread DEBUG os_win.utils.compute.vmutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Successfully changed vm state of instance-0000003b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:02.193 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Successfully changed state of VM instance-0000003b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:02.213 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpgdyu0nr1" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:02.257 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Waiting for WMI job: {'InstanceID': 'A75EE855-4FEC-48A5-82F4-3C2DB6DE9643', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15540), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:02.381 4076 3051113161480 MainThread DEBUG os_win.utils.jobutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] WMI job succeeded: {'InstanceID': 'A75EE855-4FEC-48A5-82F4-3C2DB6DE9643', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=67555), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:02.381 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'target_lun': 0, 'volume_id': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'auth_method': 'CHAP', 'auth_username': 'oi27LCgnLkajkpFVbsVr', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:02.381 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:02.515 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:02.528 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:02.584 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:02.584 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== disconnect_volume: return (202ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:02.584 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8', 'target_lun': 0, 'volume_id': 'f0f70f22-4412-4d14-80e3-17ea362502a8', 'auth_method': 'CHAP', 'auth_username': 'V3gDNm77PePMLgud3BQ5', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:02.584 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:02.709 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Setting boot order for instance: instance-0000003e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:02.709 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:02.709 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:02.777 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:02.805 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:02.805 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': 'D25E4AC2-B8B4-44EF-884B-F495E1BF2C5A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16402), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:02.805 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== disconnect_volume: return (220ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:02.805 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:02.820 4076 3051113161480 MainThread DEBUG os_win.utils.pathutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Removing directory: C:\openstack\instances\instance-0000003b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:02.820 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:02.820 4076 3051113161480 MainThread ERROR nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Traceback (most recent call last): 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(*args, **kwargs) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] admin_password, network_info, block_device_info) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(self, *args, **kwds) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] should_plug_vifs=False) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'req_state': req_state}) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] time.sleep(sleep_time) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return f(*args, **kwargs) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return self._wait_for_job(job_path) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] error_desc=err_desc) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During handling of the above exception, another exception occurred: 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Traceback (most recent call last): 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] yield resources 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] accel_info=accel_info) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(*args, **kwargs) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] admin_password, network_info, block_device_info) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(self, *args, **kwds) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] should_plug_vifs=False) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'req_state': req_state}) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] time.sleep(sleep_time) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return f(*args, **kwargs) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return self._wait_for_job(job_path) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] error_desc=err_desc) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:02.820 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:02.820 4076 3051113161480 MainThread INFO nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Terminating instance 2021-09-10 17:00:02.820 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:02.820 4076 3051113161480 MainThread INFO nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Got request to destroy instance 2021-09-10 17:00:02.841 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:02.841 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'target_lun': 0, 'volume_id': '29c54fbe-e8c1-401b-a427-b0b007cae3a7', 'auth_method': 'CHAP', 'auth_username': 'rBnpxR9o56RqMYWmjN3D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:02.852 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:02.949 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:02.959 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:02.959 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D25E4AC2-B8B4-44EF-884B-F495E1BF2C5A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=137659), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:02.959 4076 3051113293256 MainThread DEBUG os_win._utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1824953556060791 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:02.991 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:02.991 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:02.991 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:02.991 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:02.991 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-34b81967-e97c-4164-9e01-a425fa304dec - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:02.991 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-34b81967-e97c-4164-9e01-a425fa304dec - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:03.021 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-29c54fbe-e8c1-401b-a427-b0b007cae3a7 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:03.050 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== disconnect_volume: return (197ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:03.053 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'target_lun': 0, 'volume_id': '11b5cce0-1a22-4820-b4b0-bcc0b921fc84', 'auth_method': 'CHAP', 'auth_username': 'oi27LCgnLkajkpFVbsVr', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:03.053 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:03.080 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-34b81967-e97c-4164-9e01-a425fa304dec - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:03.090 4076 3051049643176 MainThread INFO nova.compute.manager [req-34b81967-e97c-4164-9e01-a425fa304dec - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:03.162 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:03.162 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-11b5cce0-1a22-4820-b4b0-bcc0b921fc84 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:03.178 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== disconnect_volume: return (124ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:03.178 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8', 'target_lun': 0, 'volume_id': 'f0f70f22-4412-4d14-80e3-17ea362502a8', 'auth_method': 'CHAP', 'auth_username': 'V3gDNm77PePMLgud3BQ5', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:03.178 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:03.336 4076 3051113161480 MainThread DEBUG os_win.utils.storage.diskutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:03.336 4076 3051113161480 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-f0f70f22-4412-4d14-80e3-17ea362502a8 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:03.336 4076 3051113161480 MainThread DEBUG os_brick.initiator.windows.iscsi [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] <== disconnect_volume: return (158ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:03.336 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:03.336 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "instance-0000003b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:03.336 4076 3051113161480 MainThread INFO nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Took 0.51 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:04.023 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': 'BF473D08-0B8F-411D-881C-388A270DCABA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15129), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:04.023 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9b83e078-3df1-4ec7-b6e3-d8286de84ac6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-add97417-8a27-4260-8deb-932d14156785 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:04.023 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-9b83e078-3df1-4ec7-b6e3-d8286de84ac6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:04.023 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-9b83e078-3df1-4ec7-b6e3-d8286de84ac6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:04.023 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9b83e078-3df1-4ec7-b6e3-d8286de84ac6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-add97417-8a27-4260-8deb-932d14156785 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:04.150 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:04.150 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:04.150 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:04.150 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:04.163 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BF473D08-0B8F-411D-881C-388A270DCABA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112157), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:04.163 4076 3051113293256 MainThread DEBUG os_win._utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3857996463775635 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:04.181 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-b1adeebf-549d-4e2c-bb99-b3361575c6a4 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:04.181 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-b1adeebf-549d-4e2c-bb99-b3361575c6a4 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:04.233 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-b1adeebf-549d-4e2c-bb99-b3361575c6a4 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:04.260 4076 3051113294664 MainThread INFO nova.compute.manager [req-b1adeebf-549d-4e2c-bb99-b3361575c6a4 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:05.168 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': '85A3C992-2549-4D5F-86A4-E0E98AAC7B8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14876), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:05.310 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '85A3C992-2549-4D5F-86A4-E0E98AAC7B8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112808), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:05.310 4076 3051113293256 MainThread DEBUG os_win._utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.53271484375 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:05.338 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:05.338 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:05.338 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:05.338 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:05.353 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9a46ecff-f707-4564-b84c-6ab82644205a - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:05.365 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-9a46ecff-f707-4564-b84c-6ab82644205a - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:05.407 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9a46ecff-f707-4564-b84c-6ab82644205a - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:05.432 4076 3051113294664 MainThread INFO nova.compute.manager [req-9a46ecff-f707-4564-b84c-6ab82644205a - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:06.178 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.208 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:00:06.326 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.326 4076 3051113294136 MainThread DEBUG nova.virt.hardware [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:00:06.326 4076 3051113294136 MainThread INFO nova.compute.claims [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Claim successful on node n-h1-808474-1 2021-09-10 17:00:06.335 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': 'E293F265-0964-4FB1-8D3A-C35F94575576', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15012), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-ffc6e67e-0a49-4c79-9748-12da98323372 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-ffc6e67e-0a49-4c79-9748-12da98323372 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Received event network-vif-plugged-88725264-1291-47ff-9cc0-13502b9a8801 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Processing event network-vif-plugged-88725264-1291-47ff-9cc0-13502b9a8801 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.461 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-169ecebc-6f5b-49fd-a015-c7003560d059 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:06.476 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:06.476 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.476 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:06.476 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:06.476 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E293F265-0964-4FB1-8D3A-C35F94575576', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:06.476 4076 3051113293256 MainThread DEBUG os_win._utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6994354724884033 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:06.500 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-05f680e6-0a56-4812-9bc8-56aa1b2fc388 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:06.500 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-05f680e6-0a56-4812-9bc8-56aa1b2fc388 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:06.500 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:06.515 4076 3051113292024 MainThread INFO nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Using config drive for instance 2021-09-10 17:00:06.529 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:06.552 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-05f680e6-0a56-4812-9bc8-56aa1b2fc388 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:06.591 4076 3051113294664 MainThread INFO nova.compute.manager [req-05f680e6-0a56-4812-9bc8-56aa1b2fc388 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:06.670 4076 3051113292024 MainThread INFO nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Creating config drive at C:\openstack\instances\instance-00000041\configdrive.iso 2021-09-10 17:00:06.708 4076 3051113292024 MainThread DEBUG oslo_concurrency.processutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000041\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpbnlcncy2 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:06.924 4076 3051113294136 MainThread DEBUG nova.compute.provider_tree [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:06.943 4076 3051113294136 MainThread DEBUG nova.scheduler.client.report [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:06.943 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:06.943 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:00:07.019 4076 3051113294136 MainThread DEBUG nova.compute.utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:00:07.019 4076 3051113294488 GreenThread-55 DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:00:07.019 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:00:07.059 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:00:07.059 4076 3051113292024 MainThread DEBUG oslo_concurrency.processutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000041\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpbnlcncy2" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:07.194 4076 3051113294488 GreenThread-55 DEBUG nova.policy [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '42267190de3648eea52604e6b2444d20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6ca07d9dc99a4581a698a232ffa065db', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:00:07.262 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:00:07.277 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Spawning new instance 2021-09-10 17:00:07.277 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:07.277 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:07.292 4076 3051113294136 MainThread DEBUG nova.block_device [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:00:07.292 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:07.292 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:07.292 4076 3051113294136 MainThread DEBUG os_win.utils.pathutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating directory: C:\openstack\instances\instance-00000044 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:00:07.292 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000044\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:00:07.318 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Resizing VHD C:\openstack\instances\instance-00000044\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:00:07.499 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': 'DCC103E9-774B-4541-A104-33AF92D0EC0C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15504), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:07.538 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Setting boot order for instance: instance-00000041: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:07.575 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:07.618 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': '2305B1AF-BE3E-4A09-A990-C4EBFDE2F506', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20797), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:07.641 4076 3051113294312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:07.641 4076 3051113294312 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:07.641 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:07.641 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:07.646 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DCC103E9-774B-4541-A104-33AF92D0EC0C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111707), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:07.646 4076 3051113293256 MainThread DEBUG os_win._utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.868834495544434 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:07.662 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-598fce74-3a8f-4b84-9acf-1df10f350655 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:07.662 4076 3051113294312 MainThread DEBUG nova.virt.hyperv.vmops [req-598fce74-3a8f-4b84-9acf-1df10f350655 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:07.724 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-598fce74-3a8f-4b84-9acf-1df10f350655 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:07.742 4076 3051113294312 MainThread INFO nova.compute.manager [req-598fce74-3a8f-4b84-9acf-1df10f350655 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:07.763 4076 3051113294312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:07.763 4076 3051113294312 MainThread INFO nova.compute.manager [-] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] VM Stopped (Lifecycle Event) 2021-09-10 17:00:07.763 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:07.763 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:07.763 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2305B1AF-BE3E-4A09-A990-C4EBFDE2F506', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122017), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:07.763 4076 3051113292024 MainThread DEBUG os_win._utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18834376335144043 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:07.788 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-b1bb8455-e586-4140-8ee7-3209102c8e54 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:07.788 4076 3051113294312 MainThread DEBUG nova.virt.hyperv.vmops [req-b1bb8455-e586-4140-8ee7-3209102c8e54 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:07.821 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-b1bb8455-e586-4140-8ee7-3209102c8e54 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:07.864 4076 3051113294312 MainThread INFO nova.compute.manager [req-b1bb8455-e586-4140-8ee7-3209102c8e54 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:08.114 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Successfully created port: f9c6f4bd-5e92-4933-9603-748e36371cfe _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:00:08.673 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': '41867F07-45D8-472A-8005-B8579EBE9370', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14543), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Received event network-vif-plugged-14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Processing event network-vif-plugged-14fcfe75-bd5b-4648-81e1-e5b0a8862bd7 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-plugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Processing event network-vif-plugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Received event network-vif-plugged-e54ef2bd-544c-413d-8411-9b83ad7fc56c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Processing event network-vif-plugged-e54ef2bd-544c-413d-8411-9b83ad7fc56c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-b083b498-e447-45f8-9086-ac65ceea526a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-b083b498-e447-45f8-9086-ac65ceea526a pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.683 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-b083b498-e447-45f8-9086-ac65ceea526a for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-46dd9dca-3150-45cc-9a35-bc1230d1380e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.683 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-46dd9dca-3150-45cc-9a35-bc1230d1380e pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.703 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-46dd9dca-3150-45cc-9a35-bc1230d1380e for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.703 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-bb905c29-b72e-4c4c-b524-e08c9df4bbd8 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.703 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-4aebdc4f-3a01-4f71-aae2-147c0d08eeb1 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-b922a519-3efc-4024-abfb-de5c20d9d578 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-b922a519-3efc-4024-abfb-de5c20d9d578 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.703 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-b922a519-3efc-4024-abfb-de5c20d9d578 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-7210f512-68cf-4fad-b1ca-29f66e80f530 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.703 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-7210f512-68cf-4fad-b1ca-29f66e80f530 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:08.703 4076 3051113294312 MainThread WARNING nova.compute.manager [req-aebb3552-93f1-4528-9cd5-327d85a894f4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-7210f512-68cf-4fad-b1ca-29f66e80f530 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:08.756 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Using config drive for instance 2021-09-10 17:00:08.773 4076 3051113293784 MainThread INFO nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Using config drive for instance 2021-09-10 17:00:08.803 4076 3051113162008 MainThread INFO nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Using config drive for instance 2021-09-10 17:00:08.803 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': '09DA521C-6A79-4DC3-A739-DAA5FA0F118E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=33400), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:08.819 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.819 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.819 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:08.819 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] VM Stopped (Lifecycle Event) 2021-09-10 17:00:08.853 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-0f19b3ff-12df-40ee-9a14-8667a44398f0 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:08.872 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-0f19b3ff-12df-40ee-9a14-8667a44398f0 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:08.872 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003e\' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)instance-0000003eDECD52C6-4EB8-4448-BC3D-424020D056A1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '41867F07-45D8-472A-8005-B8579EBE9370', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112386), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\nNot enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)\n\n'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1)", 'ErrorSummaryDescription': "'instance-0000003e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:08.872 4076 3051113293256 MainThread ERROR nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Failed to change vm state of instance-0000003e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. 2021-09-10 17:00:08.872 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Got request to destroy instance 2021-09-10 17:00:08.911 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-0f19b3ff-12df-40ee-9a14-8667a44398f0 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:08.931 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:08.931 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:08.931 4076 3051113294312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:08.931 4076 3051113294312 MainThread INFO nova.compute.manager [-] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] VM Stopped (Lifecycle Event) 2021-09-10 17:00:08.952 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '09DA521C-6A79-4DC3-A739-DAA5FA0F118E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117038), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:08.952 4076 3051113292024 MainThread DEBUG os_win._utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3772599697113037 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:08.974 4076 3051113293960 MainThread INFO nova.compute.manager [req-0f19b3ff-12df-40ee-9a14-8667a44398f0 - - - - -] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:08.974 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-a8da491c-2f89-4cc8-8080-a78e7fb50450 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:08.974 4076 3051113294312 MainThread DEBUG nova.virt.hyperv.vmops [req-a8da491c-2f89-4cc8-8080-a78e7fb50450 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:09.024 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Successfully updated port: f9c6f4bd-5e92-4933-9603-748e36371cfe _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:00:09.024 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-a8da491c-2f89-4cc8-8080-a78e7fb50450 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:09.053 4076 3051113294488 GreenThread-55 DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Acquired lock "refresh_cache-4cd8ee20-649d-4e14-8c26-085e78ef091d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:09.053 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:00:09.053 4076 3051113162008 MainThread INFO nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Creating config drive at C:\openstack\instances\instance-0000003f\configdrive.iso 2021-09-10 17:00:09.099 4076 3051113162008 MainThread DEBUG oslo_concurrency.processutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpqm07mxio execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:09.264 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Creating config drive at C:\openstack\instances\instance-00000043\configdrive.iso 2021-09-10 17:00:09.301 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000043\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy5mvemyi execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:09.444 4076 3051113293784 MainThread INFO nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Creating config drive at C:\openstack\instances\instance-00000042\configdrive.iso 2021-09-10 17:00:09.475 4076 3051113293784 MainThread DEBUG oslo_concurrency.processutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000042\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4ut_3uny execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:09.620 4076 3051113294312 MainThread INFO nova.compute.manager [req-a8da491c-2f89-4cc8-8080-a78e7fb50450 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:09.620 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Received event network-changed-f9c6f4bd-5e92-4933-9603-748e36371cfe external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:09.620 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Refreshing instance network info cache due to event network-changed-f9c6f4bd-5e92-4933-9603-748e36371cfe. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:00:09.620 4076 3051113162008 MainThread DEBUG oslo_concurrency.processutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000003f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpqm07mxio" returned: 0 in 0.516s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:09.646 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000043\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy5mvemyi" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:09.766 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:00:09.786 4076 3051113293784 MainThread DEBUG oslo_concurrency.processutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000042\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4ut_3uny" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:09.994 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': '1FBFF6C7-5F80-4B5D-9198-13BFEDA5396B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=27223), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:10.118 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Setting boot order for instance: instance-0000003f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:10.118 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Setting boot order for instance: instance-00000043: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:10.148 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1FBFF6C7-5F80-4B5D-9198-13BFEDA5396B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124523), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:10.148 4076 3051113292024 MainThread DEBUG os_win._utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.573256731033325 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:10.182 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:10.182 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:10.217 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': '0061BAE6-FAF5-41F4-B993-0EE68363E995', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20021), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:10.229 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': '0166AE1C-D02A-45B9-8CB5-FE2B094F75CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20835), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:10.293 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-00000042: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:10.352 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:10.391 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'C3090B2A-060F-4D3C-BE39-84361234101A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16216), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:10.599 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0061BAE6-FAF5-41F4-B993-0EE68363E995', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=324412), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:10.599 4076 3051113162008 MainThread DEBUG os_win._utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.4164004325866699 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:10.599 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:10.599 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:10.599 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:10.599 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:10.630 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-655b9b96-167c-4e55-831f-1fbe0105218e - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:10.630 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-655b9b96-167c-4e55-831f-1fbe0105218e - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:10.693 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:10.693 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:10.693 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:10.693 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:10.693 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-655b9b96-167c-4e55-831f-1fbe0105218e - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:10.693 4076 3051113294488 GreenThread-55 DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Updating instance_info_cache with network_info: [{"id": "f9c6f4bd-5e92-4933-9603-748e36371cfe", "address": "fa:16:3e:03:c6:e6", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf9c6f4bd-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:10.724 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-48df37e8-f6ff-40f8-9c5e-510bc0abc0c2 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:10.724 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-48df37e8-f6ff-40f8-9c5e-510bc0abc0c2 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:10.739 4076 3051093170312 MainThread INFO nova.compute.manager [req-655b9b96-167c-4e55-831f-1fbe0105218e - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:10.739 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0166AE1C-D02A-45B9-8CB5-FE2B094F75CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=448460), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:10.739 4076 3051113294840 MainThread DEBUG os_win._utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.5571022033691406 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:10.739 4076 3051113294488 GreenThread-55 DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Releasing lock "refresh_cache-4cd8ee20-649d-4e14-8c26-085e78ef091d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:10.739 4076 3051113294488 GreenThread-55 DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Instance network_info: |[{"id": "f9c6f4bd-5e92-4933-9603-748e36371cfe", "address": "fa:16:3e:03:c6:e6", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf9c6f4bd-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:00:10.739 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Preparing to wait for external event network-vif-plugged-f9c6f4bd-5e92-4933-9603-748e36371cfe prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:00:10.739 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:10.739 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:10.739 4076 3051113294136 MainThread DEBUG os_win.utils.compute.vmutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Creating VM instance-00000044 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:00:10.739 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4cd8ee20-649d-4e14-8c26-085e78ef091d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:10.739 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Refreshing network info cache for port f9c6f4bd-5e92-4933-9603-748e36371cfe _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:00:10.776 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-48df37e8-f6ff-40f8-9c5e-510bc0abc0c2 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:10.806 4076 3051093169784 MainThread INFO nova.compute.manager [req-48df37e8-f6ff-40f8-9c5e-510bc0abc0c2 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:10.806 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:10.806 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:10.806 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:10.806 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:10.834 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-0fadec07-5963-47e9-bb56-ac3f71169ec5 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:10.834 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-0fadec07-5963-47e9-bb56-ac3f71169ec5 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:10.889 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C3090B2A-060F-4D3C-BE39-84361234101A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=397178), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:10.889 4076 3051113293784 MainThread DEBUG os_win._utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.5362975597381592 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:10.889 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-0fadec07-5963-47e9-bb56-ac3f71169ec5 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:10.911 4076 3051093169784 MainThread INFO nova.compute.manager [req-0fadec07-5963-47e9-bb56-ac3f71169ec5 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:11.172 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': '4DCBFBC5-191E-42A7-9448-5149F501EBDD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14377), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:11.274 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-238fda01-e9d8-4c2d-a7f3-d498577a9dea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received event network-vif-unplugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:11.274 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-238fda01-e9d8-4c2d-a7f3-d498577a9dea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.274 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [req-238fda01-e9d8-4c2d-a7f3-d498577a9dea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:11.274 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-238fda01-e9d8-4c2d-a7f3-d498577a9dea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] No waiting events found dispatching network-vif-unplugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:11.274 4076 3051093169784 MainThread WARNING nova.compute.manager [req-238fda01-e9d8-4c2d-a7f3-d498577a9dea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Received unexpected event network-vif-unplugged-1bdbd4e2-e265-4296-84d6-9a60cc0ff9f1 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:11.304 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.304 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:11.304 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:11.324 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] VM Stopped (Lifecycle Event) 2021-09-10 17:00:11.337 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4DCBFBC5-191E-42A7-9448-5149F501EBDD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114578), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:11.337 4076 3051113292024 MainThread DEBUG os_win._utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7620835304260254 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:11.337 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-66ccce32-4e4d-456f-b30e-fa9db66e9b53 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:11.337 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-66ccce32-4e4d-456f-b30e-fa9db66e9b53 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:11.380 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-66ccce32-4e4d-456f-b30e-fa9db66e9b53 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:11.408 4076 3051093171016 MainThread INFO nova.compute.manager [req-66ccce32-4e4d-456f-b30e-fa9db66e9b53 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:11.491 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:11.491 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.491 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:11.520 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Successfully changed vm state of instance-0000003e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:11.520 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Successfully changed state of VM instance-0000003e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:11.567 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Waiting for WMI job: {'InstanceID': 'F98CB288-E443-40EF-BDD8-B7A90756BD7D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14291), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:11.618 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': 'F0FCFF76-893E-40C2-AC07-3FE6199DB899', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15605), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:11.645 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Updated VIF entry in instance network info cache for port f9c6f4bd-5e92-4933-9603-748e36371cfe. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:00:11.645 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Updating instance_info_cache with network_info: [{"id": "f9c6f4bd-5e92-4933-9603-748e36371cfe", "address": "fa:16:3e:03:c6:e6", "network": {"id": "dcfd5ed7-c94f-4b68-b56f-95b52ddf921f", "bridge": null, "label": "tempest-ImagesTestJSON-1170675645-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "6ca07d9dc99a4581a698a232ffa065db", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf9c6f4bd-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:11.648 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-f7792a72-d118-4dd8-a783-9d1866d702db 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4cd8ee20-649d-4e14-8c26-085e78ef091d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:11.685 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] WMI job succeeded: {'InstanceID': 'F98CB288-E443-40EF-BDD8-B7A90756BD7D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28445), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:11.685 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.685 4076 3051113293256 MainThread DEBUG os_win.utils.pathutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Removing directory: C:\openstack\instances\instance-0000003e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:11.685 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:11.685 4076 3051113293256 MainThread ERROR nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Traceback (most recent call last): 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(*args, **kwargs) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] admin_password, network_info, block_device_info) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(self, *args, **kwds) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] should_plug_vifs=False) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'req_state': req_state}) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] time.sleep(sleep_time) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return f(*args, **kwargs) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return self._wait_for_job(job_path) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] error_desc=err_desc) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During handling of the above exception, another exception occurred: 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Traceback (most recent call last): 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] yield resources 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] accel_info=accel_info) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(*args, **kwargs) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] admin_password, network_info, block_device_info) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(self, *args, **kwds) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] should_plug_vifs=False) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'req_state': req_state}) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] time.sleep(sleep_time) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return f(*args, **kwargs) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return self._wait_for_job(job_path) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] error_desc=err_desc) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:11.685 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:11.699 4076 3051113293256 MainThread INFO nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Terminating instance 2021-09-10 17:00:11.699 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:11.699 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Got request to destroy instance 2021-09-10 17:00:11.742 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:11.742 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.742 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "instance-0000003e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:11.742 4076 3051113293256 MainThread INFO nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:11.742 4076 3051113293256 MainThread DEBUG nova.compute.claims [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:11.742 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:11.780 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': '25F866B5-8D4E-4F82-8058-956150909C2D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22161), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:11.803 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:00:11.852 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:11.868 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:11.930 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '195F5BC9-9DB0-4F99-B905-DD79DC098A6F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16831), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:12.005 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F0FCFF76-893E-40C2-AC07-3FE6199DB899', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=325997), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:12.005 4076 3051113162008 MainThread DEBUG os_win._utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.8228073120117188 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:12.026 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:12.026 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:12.040 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:12.040 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:12.052 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-04c44d01-bd27-41d9-b528-5dac63cbd158 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:12.052 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-04c44d01-bd27-41d9-b528-5dac63cbd158 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:12.101 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:12.101 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:12.101 4076 3051113291848 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:12.101 4076 3051113291848 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:12.134 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-07ac3396-832b-4a53-a231-90666a5da5f8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:12.134 4076 3051113291848 MainThread DEBUG nova.virt.hyperv.vmops [req-07ac3396-832b-4a53-a231-90666a5da5f8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:12.134 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-04c44d01-bd27-41d9-b528-5dac63cbd158 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:12.134 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:12.155 4076 3051093170312 MainThread INFO nova.compute.manager [req-04c44d01-bd27-41d9-b528-5dac63cbd158 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:12.155 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.421s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:12.155 4076 3051113293256 MainThread ERROR nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Traceback (most recent call last): 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(*args, **kwargs) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] admin_password, network_info, block_device_info) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(self, *args, **kwds) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] should_plug_vifs=False) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'req_state': req_state}) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] time.sleep(sleep_time) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return f(*args, **kwargs) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return self._wait_for_job(job_path) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] error_desc=err_desc) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] During handling of the above exception, another exception occurred: 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Traceback (most recent call last): 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] accel_info=accel_info) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(*args, **kwargs) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] admin_password, network_info, block_device_info) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return function(self, *args, **kwds) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] should_plug_vifs=False) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'req_state': req_state}) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] time.sleep(sleep_time) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self.force_reraise() 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] raise self.value 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return f(*args, **kwargs) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] return self._wait_for_job(job_path) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] error_desc=err_desc) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. 2021-09-10 17:00:12.155 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.compute.utils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Build of instance aa775114-9ae9-476f-b4ed-2b6e1846f3bc was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003e' failed to start. Not enough memory in the system to start the virtual machine instance-0000003e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003e' failed to start. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Not enough memory in the system to start the virtual machine instance-0000003e with ram size 1024 megabytes. (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) 'instance-0000003e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD52C6-4EB8-4448-BC3D-424020D056A1) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:12.166 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:12.193 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '25F866B5-8D4E-4F82-8058-956150909C2D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=301778), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:12.193 4076 3051113294840 MainThread DEBUG os_win._utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 2.010737895965576 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:12.193 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-07ac3396-832b-4a53-a231-90666a5da5f8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:12.212 4076 3051113291848 MainThread INFO nova.compute.manager [req-07ac3396-832b-4a53-a231-90666a5da5f8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:12.225 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:12.225 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:12.225 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:12.225 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:12.239 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-0a5acf01-5adc-43ce-a138-5885df3d9d36 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:12.239 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-0a5acf01-5adc-43ce-a138-5885df3d9d36 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:12.290 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-0a5acf01-5adc-43ce-a138-5885df3d9d36 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:12.315 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '195F5BC9-9DB0-4F99-B905-DD79DC098A6F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=277558), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:12.333 4076 3051113293784 MainThread DEBUG os_win._utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.9804713726043701 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:12.333 4076 3051093171016 MainThread INFO nova.compute.manager [req-0a5acf01-5adc-43ce-a138-5885df3d9d36 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:12.364 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': '3250B5B6-19B5-40A9-BF10-DD5379F9F9F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15083), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:12.489 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:12.489 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:12.489 4076 3051093171016 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:12.502 4076 3051093171016 MainThread INFO nova.compute.manager [-] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] VM Stopped (Lifecycle Event) 2021-09-10 17:00:12.502 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3250B5B6-19B5-40A9-BF10-DD5379F9F9F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111089), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:12.502 4076 3051113292024 MainThread DEBUG os_win._utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.926832675933838 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:12.521 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-8e9d30a4-3bd5-4132-81b5-db561a84a316 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:12.521 4076 3051093171016 MainThread DEBUG nova.virt.hyperv.vmops [req-8e9d30a4-3bd5-4132-81b5-db561a84a316 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:12.567 4076 3051093171016 MainThread DEBUG nova.compute.manager [req-8e9d30a4-3bd5-4132-81b5-db561a84a316 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:12.599 4076 3051093171016 MainThread INFO nova.compute.manager [req-8e9d30a4-3bd5-4132-81b5-db561a84a316 - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:13.022 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': '87B6BD63-3588-4A61-B176-D9E9B0921699', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15670), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:13.225 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': '8333A184-E748-4A07-B032-159E3FE404C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14892), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:13.277 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:13.322 4076 3051113293256 MainThread INFO nova.compute.manager [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] [instance: aa775114-9ae9-476f-b4ed-2b6e1846f3bc] Took 1.16 seconds to deallocate network for instance. 2021-09-10 17:00:13.364 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'ABA05580-8B5B-47F9-9EF5-C51867F078CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19113), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:13.401 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '87B6BD63-3588-4A61-B176-D9E9B0921699', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=313705), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:13.413 4076 3051113162008 MainThread DEBUG os_win._utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 3.2309255599975586 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:13.413 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:13.413 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:13.413 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:13.413 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:13.449 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-4fac9597-cf01-4589-ad7d-5439c1719ca7 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:13.449 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-4fac9597-cf01-4589-ad7d-5439c1719ca7 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:13.491 4076 3051093170840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:13.491 4076 3051093170840 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:13.491 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:13.491 4076 3051093169608 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:13.510 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-4fac9597-cf01-4589-ad7d-5439c1719ca7 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:13.510 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-c6e03db4-7d09-45a5-9c1e-75f2657c9693 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:13.510 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-c6e03db4-7d09-45a5-9c1e-75f2657c9693 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:13.510 4076 3051113293256 MainThread INFO nova.scheduler.client.report [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Deleted allocations for instance aa775114-9ae9-476f-b4ed-2b6e1846f3bc 2021-09-10 17:00:13.540 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8333A184-E748-4A07-B032-159E3FE404C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=244663), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:13.540 4076 3051113294840 MainThread DEBUG os_win._utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 3.3577396869659424 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:13.540 4076 3051093169784 MainThread INFO nova.compute.manager [req-4fac9597-cf01-4589-ad7d-5439c1719ca7 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:13.554 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': 'E4BDB7E1-C108-4D72-85F3-C19C6C3D5837', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=31866), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:13.573 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-8cb1f46a-0d56-49c9-86a8-a862fc93390a 0db4ede2f101460f9ab63db4d0d76a9a 912aa6945f734abbbd12a58348a3bbbb - default default] Lock "aa775114-9ae9-476f-b4ed-2b6e1846f3bc" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.922s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:13.573 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-c6e03db4-7d09-45a5-9c1e-75f2657c9693 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:13.614 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:13.614 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:13.614 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:13.614 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:13.620 4076 3051093170840 MainThread INFO nova.compute.manager [req-c6e03db4-7d09-45a5-9c1e-75f2657c9693 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:13.620 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-5da842fd-42be-48f0-9363-de39f64669a3 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:13.640 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-5da842fd-42be-48f0-9363-de39f64669a3 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:13.655 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'ABA05580-8B5B-47F9-9EF5-C51867F078CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=208444), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:13.655 4076 3051113293784 MainThread DEBUG os_win._utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 3.3027498722076416 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:13.684 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-5da842fd-42be-48f0-9363-de39f64669a3 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:13.713 4076 3051093169784 MainThread INFO nova.compute.manager [req-5da842fd-42be-48f0-9363-de39f64669a3 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:13.727 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:13.727 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] VM Stopped (Lifecycle Event) 2021-09-10 17:00:13.727 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:13.727 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:13.756 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-5a63eb21-bb01-4004-a7f6-6887cf852fae - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:13.756 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-5a63eb21-bb01-4004-a7f6-6887cf852fae - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:13.800 4076 3051113292024 MainThread ERROR os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000041\' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000041\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)instance-00000041419DE39C-EF91-442E-837C-EC63177DD5BC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E4BDB7E1-C108-4D72-85F3-C19C6C3D5837', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=167944), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\nNot enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)\n\n'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC)", 'ErrorSummaryDescription': "'instance-00000041' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000041.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:13.800 4076 3051113292024 MainThread ERROR nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Failed to change vm state of instance-00000041 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. 2021-09-10 17:00:13.800 4076 3051113292024 MainThread INFO nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Got request to destroy instance 2021-09-10 17:00:13.800 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-5a63eb21-bb01-4004-a7f6-6887cf852fae - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:13.823 4076 3051093169784 MainThread INFO nova.compute.manager [req-5a63eb21-bb01-4004-a7f6-6887cf852fae - - - - -] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:14.446 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': '19730710-8B16-455C-B767-F35A973AE4B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16532), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:14.561 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': 'C396D3B5-6964-4053-BD40-EB8E3D9A16F0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15738), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:14.696 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '93CDC29B-AF6F-4ACF-9577-38D43913FE5A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16724), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:14.870 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '19730710-8B16-455C-B767-F35A973AE4B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=330046), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:14.870 4076 3051113162008 MainThread DEBUG os_win._utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.687373161315918 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:14.870 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:14.870 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:14.870 4076 3051093169784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:14.870 4076 3051093169784 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:14.911 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-d7c5c326-2868-458c-b295-be7c59bdc8fe - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:14.911 4076 3051093169784 MainThread DEBUG nova.virt.hyperv.vmops [req-d7c5c326-2868-458c-b295-be7c59bdc8fe - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:14.947 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:14.947 4076 3051093171016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:14.947 4076 3051093170840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:14.947 4076 3051093170840 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:14.960 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-26e7b98a-fbc6-4936-8dee-7afdfe840ffe - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:14.960 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-26e7b98a-fbc6-4936-8dee-7afdfe840ffe - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:14.960 4076 3051113161480 MainThread INFO nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Took 11.62 seconds to detach 3 volumes for instance. 2021-09-10 17:00:14.960 4076 3051113161480 MainThread DEBUG nova.compute.claims [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:14.960 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:14.960 4076 3051093169784 MainThread DEBUG nova.compute.manager [req-d7c5c326-2868-458c-b295-be7c59bdc8fe - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:14.990 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C396D3B5-6964-4053-BD40-EB8E3D9A16F0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=352399), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:14.990 4076 3051113294840 MainThread DEBUG os_win._utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.807780504226685 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:14.990 4076 3051093169784 MainThread INFO nova.compute.manager [req-d7c5c326-2868-458c-b295-be7c59bdc8fe - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:15.021 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-26e7b98a-fbc6-4936-8dee-7afdfe840ffe - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:15.038 4076 3051093170840 MainThread INFO nova.compute.manager [req-26e7b98a-fbc6-4936-8dee-7afdfe840ffe - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:15.071 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:15.071 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:15.071 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:15.071 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:15.091 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.hostops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:15.091 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-36ea4430-7e4f-42df-8594-718e8092e6ff - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:15.091 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-36ea4430-7e4f-42df-8594-718e8092e6ff - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:15.102 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '93CDC29B-AF6F-4ACF-9577-38D43913FE5A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=343597), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:15.102 4076 3051113293784 MainThread DEBUG os_win._utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 4.749995231628418 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:15.124 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.hostops [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:15.147 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-36ea4430-7e4f-42df-8594-718e8092e6ff - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:15.180 4076 3051049643176 MainThread INFO nova.compute.manager [req-36ea4430-7e4f-42df-8594-718e8092e6ff - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:15.336 4076 3051113161480 MainThread DEBUG nova.compute.provider_tree [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:15.353 4076 3051113161480 MainThread DEBUG nova.scheduler.client.report [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:15.365 4076 3051113161480 MainThread ERROR nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Traceback (most recent call last): 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(*args, **kwargs) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] admin_password, network_info, block_device_info) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(self, *args, **kwds) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] should_plug_vifs=False) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'req_state': req_state}) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] time.sleep(sleep_time) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return f(*args, **kwargs) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return self._wait_for_job(job_path) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] error_desc=err_desc) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] During handling of the above exception, another exception occurred: 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Traceback (most recent call last): 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] accel_info=accel_info) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(*args, **kwargs) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] admin_password, network_info, block_device_info) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return function(self, *args, **kwds) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] should_plug_vifs=False) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'req_state': req_state}) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] time.sleep(sleep_time) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self.force_reraise() 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] raise self.value 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return f(*args, **kwargs) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] return self._wait_for_job(job_path) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] error_desc=err_desc) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. 2021-09-10 17:00:15.365 4076 3051113161480 MainThread TRACE nova.compute.manager [instance: b0745679-4000-4aff-bd0a-29ded4711f24] 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.compute.utils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Build of instance b0745679-4000-4aff-bd0a-29ded4711f24 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003b' failed to start. Not enough memory in the system to start the virtual machine instance-0000003b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003b' failed to start. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Not enough memory in the system to start the virtual machine instance-0000003b with ram size 1024 megabytes. (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) 'instance-0000003b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4E1683C8-BA57-4F8A-85B2-F9C7195A5DF8) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:15.365 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:15.779 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:15.779 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:15.779 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:15.779 4076 3051113292024 MainThread DEBUG os_win.utils.compute.vmutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Successfully changed vm state of instance-00000041 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:15.779 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Successfully changed state of VM instance-00000041 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:15.833 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Waiting for WMI job: {'InstanceID': 'B2596C19-5EBF-4B0E-B7F4-9F06029D44F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13843), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:15.884 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': 'CA0234DA-49CC-4A5B-821B-B0CFD2422538', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14898), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:15.943 4076 3051113292024 MainThread DEBUG os_win.utils.jobutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] WMI job succeeded: {'InstanceID': 'B2596C19-5EBF-4B0E-B7F4-9F06029D44F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28945), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:15.943 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:15.943 4076 3051113292024 MainThread DEBUG os_win.utils.pathutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Removing directory: C:\openstack\instances\instance-00000041 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:15.943 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:15.943 4076 3051113292024 MainThread ERROR nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Traceback (most recent call last): 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(*args, **kwargs) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] admin_password, network_info, block_device_info) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(self, *args, **kwds) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] should_plug_vifs=False) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'req_state': req_state}) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] time.sleep(sleep_time) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return f(*args, **kwargs) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return self._wait_for_job(job_path) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] error_desc=err_desc) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During handling of the above exception, another exception occurred: 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Traceback (most recent call last): 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] yield resources 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] accel_info=accel_info) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(*args, **kwargs) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] admin_password, network_info, block_device_info) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(self, *args, **kwds) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] should_plug_vifs=False) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'req_state': req_state}) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] time.sleep(sleep_time) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return f(*args, **kwargs) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return self._wait_for_job(job_path) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] error_desc=err_desc) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:15.943 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:15.966 4076 3051113292024 MainThread INFO nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Terminating instance 2021-09-10 17:00:15.966 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:15.966 4076 3051113292024 MainThread INFO nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Got request to destroy instance 2021-09-10 17:00:15.974 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:15.974 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:15.974 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "instance-00000041" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:15.974 4076 3051113292024 MainThread INFO nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:15.974 4076 3051113292024 MainThread DEBUG nova.compute.claims [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:15.990 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:16.030 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': '57B42E19-0D9F-4D9B-A530-D909F4D64648', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15030), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:16.086 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.hostops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:16.104 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.hostops [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:16.116 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '657B44D0-221E-460A-8EF2-F8AAEDCFC0C1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15673), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:16.272 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CA0234DA-49CC-4A5B-821B-B0CFD2422538', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=320642), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:16.272 4076 3051113162008 MainThread DEBUG os_win._utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 6.090178728103638 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:16.272 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:16.272 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:16.272 4076 3051113161656 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:16.272 4076 3051113161656 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:16.308 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-e3dc4b48-1227-44bb-a89a-c9c8fa06469c - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:16.308 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-e3dc4b48-1227-44bb-a89a-c9c8fa06469c - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:16.342 4076 3051113292024 MainThread DEBUG nova.compute.provider_tree [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:16.351 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:16.351 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:16.351 4076 3051113293256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:16.351 4076 3051113293256 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:16.351 4076 3051113292024 MainThread DEBUG nova.scheduler.client.report [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:16.364 4076 3051113292024 MainThread ERROR nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Traceback (most recent call last): 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(*args, **kwargs) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] admin_password, network_info, block_device_info) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(self, *args, **kwds) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] should_plug_vifs=False) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'req_state': req_state}) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] time.sleep(sleep_time) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return f(*args, **kwargs) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return self._wait_for_job(job_path) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] error_desc=err_desc) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] During handling of the above exception, another exception occurred: 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Traceback (most recent call last): 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] accel_info=accel_info) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(*args, **kwargs) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] admin_password, network_info, block_device_info) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return function(self, *args, **kwds) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] should_plug_vifs=False) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'req_state': req_state}) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] time.sleep(sleep_time) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self.force_reraise() 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] raise self.value 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return f(*args, **kwargs) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] return self._wait_for_job(job_path) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] error_desc=err_desc) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. 2021-09-10 17:00:16.364 4076 3051113292024 MainThread TRACE nova.compute.manager [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.compute.utils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Build of instance 5b7f9971-39c8-4512-b873-8d2e09fd6941 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000041' failed to start. Not enough memory in the system to start the virtual machine instance-00000041. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000041' failed to start. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Not enough memory in the system to start the virtual machine instance-00000041 with ram size 1024 megabytes. (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) 'instance-00000041' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 419DE39C-EF91-442E-837C-EC63177DD5BC) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:16.364 4076 3051113292024 MainThread DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:16.364 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-e3dc4b48-1227-44bb-a89a-c9c8fa06469c - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:16.387 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-9e9a5010-6ae1-41e7-b098-e46b094fc8a8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:16.387 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-9e9a5010-6ae1-41e7-b098-e46b094fc8a8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:16.399 4076 3051113161656 MainThread INFO nova.compute.manager [req-e3dc4b48-1227-44bb-a89a-c9c8fa06469c - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:16.419 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '57B42E19-0D9F-4D9B-A530-D909F4D64648', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=305879), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:16.419 4076 3051113294840 MainThread DEBUG os_win._utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 6.236523151397705 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:16.436 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-9e9a5010-6ae1-41e7-b098-e46b094fc8a8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:16.449 4076 3051113293256 MainThread INFO nova.compute.manager [req-9e9a5010-6ae1-41e7-b098-e46b094fc8a8 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:16.473 4076 3051113293256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:16.473 4076 3051113293256 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:16.473 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:16.473 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:16.497 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-937a3cd0-c17a-4ca2-816d-8133639a0915 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:16.497 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-937a3cd0-c17a-4ca2-816d-8133639a0915 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:16.553 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-937a3cd0-c17a-4ca2-816d-8133639a0915 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:16.553 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '657B44D0-221E-460A-8EF2-F8AAEDCFC0C1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=315827), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:16.553 4076 3051113293784 MainThread DEBUG os_win._utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 6.200512170791626 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:16.573 4076 3051113293256 MainThread INFO nova.compute.manager [req-937a3cd0-c17a-4ca2-816d-8133639a0915 - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:17.288 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': 'BF851543-A807-4EBF-BE19-65611B042288', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15640), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:17.428 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': 'E0BC0EF8-CB1E-42AF-9913-A70A2090E49B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17785), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:17.578 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '10BC7AB3-0336-42A4-8676-9AFF82A7F708', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15496), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:17.645 4076 3051113162008 MainThread ERROR os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000003f\' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000003f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)instance-0000003fAF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BF851543-A807-4EBF-BE19-65611B042288', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=309798), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\nNot enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)\n\n'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E)", 'ErrorSummaryDescription': "'instance-0000003f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000003f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:17.655 4076 3051113162008 MainThread ERROR nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Failed to change vm state of instance-0000003f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. 2021-09-10 17:00:17.655 4076 3051113162008 MainThread INFO nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Got request to destroy instance 2021-09-10 17:00:17.683 4076 3051113161656 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:17.683 4076 3051113161656 MainThread INFO nova.compute.manager [-] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] VM Stopped (Lifecycle Event) 2021-09-10 17:00:17.683 4076 3051112928008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:17.683 4076 3051112928008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:17.709 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-4ff1e9c5-eda5-46ed-a9e6-ee3e09dac9c8 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:17.709 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-4ff1e9c5-eda5-46ed-a9e6-ee3e09dac9c8 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:17.759 4076 3051112927656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:17.759 4076 3051112927656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:17.759 4076 3051112928008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:17.759 4076 3051112928008 MainThread INFO nova.compute.manager [-] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] VM Stopped (Lifecycle Event) 2021-09-10 17:00:17.772 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-4ff1e9c5-eda5-46ed-a9e6-ee3e09dac9c8 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:17.772 4076 3051112928008 MainThread DEBUG nova.compute.manager [req-a2a47854-a4b9-49e3-89e1-62fd633890e0 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:17.772 4076 3051112928008 MainThread DEBUG nova.virt.hyperv.vmops [req-a2a47854-a4b9-49e3-89e1-62fd633890e0 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:17.798 4076 3051113161656 MainThread INFO nova.compute.manager [req-4ff1e9c5-eda5-46ed-a9e6-ee3e09dac9c8 - - - - -] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:17.810 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000043\' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000043\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)instance-00000043FCA39751-A539-4326-A969-0884589CE70C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E0BC0EF8-CB1E-42AF-9913-A70A2090E49B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=287075), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\nNot enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)\n\n'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C)", 'ErrorSummaryDescription': "'instance-00000043' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000043.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:17.810 4076 3051113294840 MainThread ERROR nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Failed to change vm state of instance-00000043 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. 2021-09-10 17:00:17.810 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Got request to destroy instance 2021-09-10 17:00:17.839 4076 3051112928008 MainThread DEBUG nova.compute.manager [req-a2a47854-a4b9-49e3-89e1-62fd633890e0 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:17.871 4076 3051112928008 MainThread INFO nova.compute.manager [req-a2a47854-a4b9-49e3-89e1-62fd633890e0 - - - - -] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:17.881 4076 3051112928008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:17.881 4076 3051112928008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:17.881 4076 3051113161656 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:17.881 4076 3051113161656 MainThread INFO nova.compute.manager [-] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] VM Stopped (Lifecycle Event) 2021-09-10 17:00:17.906 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-222d53fa-579a-488b-94c6-6f645132c44c - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:17.906 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-222d53fa-579a-488b-94c6-6f645132c44c - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:17.974 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-222d53fa-579a-488b-94c6-6f645132c44c - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:17.974 4076 3051113293784 MainThread ERROR os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000042\' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000042\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)instance-000000426B57C303-7551-482A-B8E7-6639672C727C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '10BC7AB3-0336-42A4-8676-9AFF82A7F708', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=286392), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\nNot enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)\n\n'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C)", 'ErrorSummaryDescription': "'instance-00000042' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000042.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:17.974 4076 3051113293784 MainThread ERROR nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-00000042 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. 2021-09-10 17:00:17.974 4076 3051113293784 MainThread INFO nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Got request to destroy instance 2021-09-10 17:00:17.996 4076 3051113161656 MainThread INFO nova.compute.manager [req-222d53fa-579a-488b-94c6-6f645132c44c - - - - -] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:18.224 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-21b815b2-c389-4db9-bfaa-81978fd8dd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Received event network-vif-plugged-f9c6f4bd-5e92-4933-9603-748e36371cfe external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:18.224 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-21b815b2-c389-4db9-bfaa-81978fd8dd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:18.224 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [req-21b815b2-c389-4db9-bfaa-81978fd8dd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:18.224 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-21b815b2-c389-4db9-bfaa-81978fd8dd9c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Processing event network-vif-plugged-f9c6f4bd-5e92-4933-9603-748e36371cfe _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:18.279 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Using config drive for instance 2021-09-10 17:00:18.466 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Creating config drive at C:\openstack\instances\instance-00000044\configdrive.iso 2021-09-10 17:00:18.496 4076 3051113294136 MainThread DEBUG oslo_concurrency.processutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000044\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphigw3olv execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:18.740 4076 3051113292024 MainThread DEBUG nova.network.neutron [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:18.778 4076 3051113292024 MainThread INFO nova.compute.manager [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] [instance: 5b7f9971-39c8-4512-b873-8d2e09fd6941] Took 2.41 seconds to deallocate network for instance. 2021-09-10 17:00:18.825 4076 3051113294136 MainThread DEBUG oslo_concurrency.processutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000044\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphigw3olv" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:19.038 4076 3051113292024 MainThread INFO nova.scheduler.client.report [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Deleted allocations for instance 5b7f9971-39c8-4512-b873-8d2e09fd6941 2021-09-10 17:00:19.083 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-774d124b-6a08-4af6-8bfd-31c18ba97fb9 bab205b0b6fc4ec2b4ffc7f2fdf0a14e 966c4d4813584979a85cc2a7055682b3 - default default] Lock "5b7f9971-39c8-4512-b873-8d2e09fd6941" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 36.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:19.289 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Setting boot order for instance: instance-00000044: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:19.354 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:19.386 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'F4931D27-3CC3-42AF-860A-1CDDE7FB29B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15708), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:19.527 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:19.527 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:19.537 4076 3051113161656 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:19.537 4076 3051113161656 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:19.537 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F4931D27-3CC3-42AF-860A-1CDDE7FB29B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117513), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:19.537 4076 3051113294136 MainThread DEBUG os_win._utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1830134391784668 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:19.537 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-35c7d37c-4401-4c76-8757-1e586e49392c - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:19.561 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [req-35c7d37c-4401-4c76-8757-1e586e49392c - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:19.600 4076 3051113161656 MainThread DEBUG nova.compute.manager [req-35c7d37c-4401-4c76-8757-1e586e49392c - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:19.637 4076 3051113161656 MainThread INFO nova.compute.manager [req-35c7d37c-4401-4c76-8757-1e586e49392c - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:20.005 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:20.005 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.005 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.037 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Successfully changed vm state of instance-00000043 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:20.037 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Successfully changed state of VM instance-00000043 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:20.093 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Waiting for WMI job: {'InstanceID': '16AC217E-F034-4379-B08C-323CB0609945', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18132), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:20.177 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:20.177 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.177 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.196 4076 3051113293784 MainThread DEBUG os_win.utils.compute.vmutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-00000042 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:20.196 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-00000042 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:20.240 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] WMI job succeeded: {'InstanceID': '16AC217E-F034-4379-B08C-323CB0609945', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29360), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:20.240 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.240 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Removing directory: C:\openstack\instances\instance-00000043 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:20.240 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.240 4076 3051113294840 MainThread ERROR nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Traceback (most recent call last): 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(*args, **kwargs) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(self, *args, **kwds) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] should_plug_vifs=False) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'req_state': req_state}) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] time.sleep(sleep_time) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return f(*args, **kwargs) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return self._wait_for_job(job_path) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] error_desc=err_desc) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During handling of the above exception, another exception occurred: 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Traceback (most recent call last): 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] yield resources 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] accel_info=accel_info) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(*args, **kwargs) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(self, *args, **kwds) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] should_plug_vifs=False) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'req_state': req_state}) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] time.sleep(sleep_time) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return f(*args, **kwargs) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return self._wait_for_job(job_path) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] error_desc=err_desc) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.240 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.240 4076 3051113294840 MainThread INFO nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Terminating instance 2021-09-10 17:00:20.240 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:20.240 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Got request to destroy instance 2021-09-10 17:00:20.240 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '192CA349-5EF4-4535-924F-FE7C7C56F625', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15275), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:20.270 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:20.270 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.270 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "instance-00000043" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.270 4076 3051113294840 MainThread INFO nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:20.270 4076 3051113294840 MainThread DEBUG nova.compute.claims [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:20.270 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.372 4076 3051113293784 MainThread DEBUG os_win.utils.jobutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': '192CA349-5EF4-4535-924F-FE7C7C56F625', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30479), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:20.372 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.372 4076 3051113293784 MainThread DEBUG os_win.utils.pathutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-00000042 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:20.372 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.372 4076 3051113293784 MainThread ERROR nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Traceback (most recent call last): 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(*args, **kwargs) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(self, *args, **kwds) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] should_plug_vifs=False) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'req_state': req_state}) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] time.sleep(sleep_time) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return f(*args, **kwargs) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return self._wait_for_job(job_path) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] error_desc=err_desc) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During handling of the above exception, another exception occurred: 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Traceback (most recent call last): 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] yield resources 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] accel_info=accel_info) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(*args, **kwargs) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(self, *args, **kwds) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] should_plug_vifs=False) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'req_state': req_state}) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] time.sleep(sleep_time) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return f(*args, **kwargs) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return self._wait_for_job(job_path) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] error_desc=err_desc) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:20.372 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:20.372 4076 3051113293784 MainThread INFO nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Terminating instance 2021-09-10 17:00:20.372 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:20.372 4076 3051113293784 MainThread INFO nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Got request to destroy instance 2021-09-10 17:00:20.372 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:20.398 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:20.398 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.398 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000042" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.398 4076 3051113293784 MainThread INFO nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:20.398 4076 3051113293784 MainThread DEBUG nova.compute.claims [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:20.412 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:20.574 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '796CAD89-C475-450A-92FB-32E4A3DE7300', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16415), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:20.623 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.646 4076 3051113294840 MainThread ERROR nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Traceback (most recent call last): 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(*args, **kwargs) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(self, *args, **kwds) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] should_plug_vifs=False) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'req_state': req_state}) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] time.sleep(sleep_time) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return f(*args, **kwargs) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return self._wait_for_job(job_path) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] error_desc=err_desc) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] During handling of the above exception, another exception occurred: 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Traceback (most recent call last): 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] accel_info=accel_info) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(*args, **kwargs) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] admin_password, network_info, block_device_info) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return function(self, *args, **kwds) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] should_plug_vifs=False) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'req_state': req_state}) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] time.sleep(sleep_time) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self.force_reraise() 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] raise self.value 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return f(*args, **kwargs) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] return self._wait_for_job(job_path) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] error_desc=err_desc) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. 2021-09-10 17:00:20.646 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Build of instance 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000043' failed to start. Not enough memory in the system to start the virtual machine instance-00000043. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000043' failed to start. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Not enough memory in the system to start the virtual machine instance-00000043 with ram size 1024 megabytes. (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) 'instance-00000043' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FCA39751-A539-4326-A969-0884589CE70C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:20.646 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:20.661 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.265s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.711 4076 3051112928888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:20.711 4076 3051112928888 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:20.726 4076 3051112929064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.726 4076 3051112929064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:20.726 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '796CAD89-C475-450A-92FB-32E4A3DE7300', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119398), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:20.726 4076 3051113294136 MainThread DEBUG os_win._utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3717169761657715 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:20.739 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-963c123d-28fa-4f31-ab02-639595de2a26 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:20.739 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-963c123d-28fa-4f31-ab02-639595de2a26 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:20.759 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.hostops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:20.788 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.hostops [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:20.788 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-963c123d-28fa-4f31-ab02-639595de2a26 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:20.810 4076 3051112928888 MainThread INFO nova.compute.manager [req-963c123d-28fa-4f31-ab02-639595de2a26 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:20.975 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:20.975 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:20.975 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:21.001 4076 3051113293784 MainThread DEBUG nova.compute.provider_tree [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:21.006 4076 3051113162008 MainThread DEBUG os_win.utils.compute.vmutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Successfully changed vm state of instance-0000003f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:21.006 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Successfully changed state of VM instance-0000003f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:21.006 4076 3051113293784 MainThread DEBUG nova.scheduler.client.report [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:21.006 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:21.006 4076 3051113293784 MainThread ERROR nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Traceback (most recent call last): 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(*args, **kwargs) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] admin_password, network_info, block_device_info) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(self, *args, **kwds) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] should_plug_vifs=False) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'req_state': req_state}) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] time.sleep(sleep_time) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return f(*args, **kwargs) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return self._wait_for_job(job_path) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] error_desc=err_desc) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] During handling of the above exception, another exception occurred: 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Traceback (most recent call last): 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] accel_info=accel_info) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(*args, **kwargs) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] admin_password, network_info, block_device_info) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return function(self, *args, **kwds) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] should_plug_vifs=False) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'req_state': req_state}) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] time.sleep(sleep_time) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self.force_reraise() 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] raise self.value 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return f(*args, **kwargs) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] return self._wait_for_job(job_path) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] error_desc=err_desc) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. 2021-09-10 17:00:21.006 4076 3051113293784 MainThread TRACE nova.compute.manager [instance: b1382ef7-107d-4113-a901-1877f9724cd1] 2021-09-10 17:00:21.006 4076 3051113293784 MainThread DEBUG nova.compute.utils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:21.029 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Build of instance b1382ef7-107d-4113-a901-1877f9724cd1 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000042' failed to start. Not enough memory in the system to start the virtual machine instance-00000042. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000042' failed to start. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Not enough memory in the system to start the virtual machine instance-00000042 with ram size 1024 megabytes. (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) 'instance-00000042' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6B57C303-7551-482A-B8E7-6639672C727C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:21.029 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:21.029 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:21.029 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:21.029 4076 3051113293784 MainThread DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:21.067 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Waiting for WMI job: {'InstanceID': '81AFA31F-BE00-4714-9231-CFD125899E4D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14998), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:21.181 4076 3051113162008 MainThread DEBUG os_win.utils.jobutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] WMI job succeeded: {'InstanceID': '81AFA31F-BE00-4714-9231-CFD125899E4D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=58595), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:21.181 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324', 'target_lun': 0, 'volume_id': '174960b8-7aa2-43c8-aff0-31d5f7de9324', 'auth_method': 'CHAP', 'auth_username': 'rYi4xVp8k8gT6zEJHGPP', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:21.195 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:21.301 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:21.317 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:21.377 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:21.377 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== disconnect_volume: return (182ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:21.377 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1', 'target_lun': 0, 'volume_id': '7687f3e5-535f-43cb-92df-79afc5c86fb1', 'auth_method': 'CHAP', 'auth_username': 'zyJxKJaxT6B3MYRsZn7s', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:21.377 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:21.505 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:21.505 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:21.576 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:21.585 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== disconnect_volume: return (208ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:21.585 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:21.585 4076 3051113162008 MainThread DEBUG os_win.utils.pathutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Removing directory: C:\openstack\instances\instance-0000003f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:21.585 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:21.585 4076 3051113162008 MainThread ERROR nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Traceback (most recent call last): 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(*args, **kwargs) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] admin_password, network_info, block_device_info) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(self, *args, **kwds) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] should_plug_vifs=False) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'req_state': req_state}) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] time.sleep(sleep_time) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return f(*args, **kwargs) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return self._wait_for_job(job_path) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] error_desc=err_desc) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During handling of the above exception, another exception occurred: 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Traceback (most recent call last): 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] yield resources 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] accel_info=accel_info) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(*args, **kwargs) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] admin_password, network_info, block_device_info) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(self, *args, **kwds) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] should_plug_vifs=False) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'req_state': req_state}) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] time.sleep(sleep_time) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return f(*args, **kwargs) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return self._wait_for_job(job_path) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] error_desc=err_desc) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:21.585 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:21.585 4076 3051113162008 MainThread INFO nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Terminating instance 2021-09-10 17:00:21.585 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:21.585 4076 3051113162008 MainThread INFO nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Got request to destroy instance 2021-09-10 17:00:21.609 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:21.609 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05', 'target_lun': 0, 'volume_id': '36413d8d-e561-4803-ae3b-516b90e60d05', 'auth_method': 'CHAP', 'auth_username': 'r7zZCaWyvbNG2TWUNX7m', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:21.609 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:21.648 4076 3051113293784 MainThread DEBUG nova.network.neutron [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:21.677 4076 3051113293784 MainThread INFO nova.compute.manager [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: b1382ef7-107d-4113-a901-1877f9724cd1] Took 0.64 seconds to deallocate network for instance. 2021-09-10 17:00:21.708 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:21.708 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:21.729 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:21.729 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:00:21.755 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'EE2685C2-FDB1-487E-AC02-3AC36853EEB5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21151), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:21.776 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-36413d8d-e561-4803-ae3b-516b90e60d05 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:00:21.776 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== disconnect_volume: return (167ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:21.776 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324', 'target_lun': 0, 'volume_id': '174960b8-7aa2-43c8-aff0-31d5f7de9324', 'auth_method': 'CHAP', 'auth_username': 'rYi4xVp8k8gT6zEJHGPP', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:21.799 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:21.849 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:21.849 4076 3051112930120 MainThread DEBUG nova.virt.hardware [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:00:21.849 4076 3051112930120 MainThread INFO nova.compute.claims [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Claim successful on node n-h1-808474-1 2021-09-10 17:00:21.849 4076 3051113293784 MainThread INFO nova.scheduler.client.report [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance b1382ef7-107d-4113-a901-1877f9724cd1 2021-09-10 17:00:21.884 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:21.903 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-174960b8-7aa2-43c8-aff0-31d5f7de9324 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:21.903 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== disconnect_volume: return (104ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:21.903 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1', 'target_lun': 0, 'volume_id': '7687f3e5-535f-43cb-92df-79afc5c86fb1', 'auth_method': 'CHAP', 'auth_username': 'zyJxKJaxT6B3MYRsZn7s', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:00:21.903 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:00:21.921 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-a2b70c9f-d640-4816-a7e5-589b74d397ad 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "b1382ef7-107d-4113-a901-1877f9724cd1" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 35.578s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:21.921 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:21.921 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:21.921 4076 3051113293784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:21.921 4076 3051113293784 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:21.921 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EE2685C2-FDB1-487E-AC02-3AC36853EEB5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=137240), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:21.921 4076 3051113294136 MainThread DEBUG os_win._utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.567275285720825 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:21.944 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-d125cdad-1028-4e68-92e0-2cafd1cedf7f - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:21.944 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-d125cdad-1028-4e68-92e0-2cafd1cedf7f - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:21.974 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.hostops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:21.989 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.hostops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:22.007 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-d125cdad-1028-4e68-92e0-2cafd1cedf7f - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:22.036 4076 3051113293784 MainThread INFO nova.compute.manager [req-d125cdad-1028-4e68-92e0-2cafd1cedf7f - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:22.036 4076 3051113162008 MainThread DEBUG os_win.utils.storage.diskutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:00:22.069 4076 3051113162008 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-7687f3e5-535f-43cb-92df-79afc5c86fb1 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:00:22.069 4076 3051113162008 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] <== disconnect_volume: return (166ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:00:22.069 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:22.069 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "instance-0000003f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:22.069 4076 3051113162008 MainThread INFO nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Took 0.49 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:22.208 4076 3051112930120 MainThread DEBUG nova.compute.provider_tree [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:22.208 4076 3051112930120 MainThread DEBUG nova.scheduler.client.report [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:22.208 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:22.208 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:00:22.317 4076 3051112930120 MainThread DEBUG nova.compute.utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:00:22.333 4076 3051112928888 GreenThread-56 DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not allocating networking since 'none' was specified. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1752 2021-09-10 17:00:22.336 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:00:22.532 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:00:22.551 4076 3051112930120 MainThread INFO nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Spawning new instance 2021-09-10 17:00:22.559 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:22.559 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:22.559 4076 3051112930120 MainThread DEBUG nova.block_device [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:00:22.559 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:22.559 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:22.573 4076 3051112930120 MainThread DEBUG os_win.utils.pathutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Creating directory: C:\openstack\instances\instance-00000045 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:00:22.573 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000045\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:00:22.573 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:22.601 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Resizing VHD C:\openstack\instances\instance-00000045\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:00:22.614 4076 3051113294840 MainThread INFO nova.compute.manager [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] [instance: 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f] Took 1.97 seconds to deallocate network for instance. 2021-09-10 17:00:22.630 4076 3051112930120 MainThread DEBUG os_win.utils.compute.vmutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Creating VM instance-00000045 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:00:22.903 4076 3051113294840 MainThread INFO nova.scheduler.client.report [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Deleted allocations for instance 0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f 2021-09-10 17:00:22.929 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '14CABE35-9297-49B1-9A13-C99D4A0D7C12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14557), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:22.958 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e09e66eb-bf79-4bdc-bb1b-99253820b6db 07d38d374c614cedbb4fed9d7dc73f45 4b1e8ca5b70b4088b2d52a886b333d20 - default default] Lock "0cccf6bf-6ec9-4777-ac92-f5c40f0d6f7f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 34.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:23.077 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:23.077 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:23.099 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:23.099 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:23.118 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '14CABE35-9297-49B1-9A13-C99D4A0D7C12', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118122), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:23.118 4076 3051113294136 MainThread DEBUG os_win._utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.764188528060913 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:23.118 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-0e4e0f62-a956-482c-a45f-ec26a548f516 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:23.137 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-0e4e0f62-a956-482c-a45f-ec26a548f516 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:23.179 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-0e4e0f62-a956-482c-a45f-ec26a548f516 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:23.209 4076 3051113294840 MainThread INFO nova.compute.manager [req-0e4e0f62-a956-482c-a45f-ec26a548f516 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:23.770 4076 3051112930120 MainThread INFO nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Using config drive for instance 2021-09-10 17:00:24.156 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '7A6B44B9-502D-4E57-8481-817DFEBA58CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14424), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:24.291 4076 3051112928712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:24.291 4076 3051112928712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:24.291 4076 3051112928888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:24.291 4076 3051112928888 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:24.307 4076 3051112930120 MainThread INFO nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Creating config drive at C:\openstack\instances\instance-00000045\configdrive.iso 2021-09-10 17:00:24.338 4076 3051112930120 MainThread DEBUG oslo_concurrency.processutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000045\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpw27eo4f5 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:24.473 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b339a48f-281b-4a18-b265-1f1c3dec74e4 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:24.473 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b339a48f-281b-4a18-b265-1f1c3dec74e4 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:24.488 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7A6B44B9-502D-4E57-8481-817DFEBA58CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112002), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:24.488 4076 3051113294136 MainThread DEBUG os_win._utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.1342933177948 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:24.530 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b339a48f-281b-4a18-b265-1f1c3dec74e4 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:24.566 4076 3051112928888 MainThread INFO nova.compute.manager [req-b339a48f-281b-4a18-b265-1f1c3dec74e4 - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:24.629 4076 3051112930120 MainThread DEBUG oslo_concurrency.processutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000045\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpw27eo4f5" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:25.020 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Setting boot order for instance: instance-00000045: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:25.067 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:25.099 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': '1F868FF7-C87B-4883-9286-D97AC7303A57', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14963), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:25.192 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:25.192 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:25.192 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:25.192 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:25.248 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1F868FF7-C87B-4883-9286-D97AC7303A57', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=93070), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:25.248 4076 3051112930120 MainThread DEBUG os_win._utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1810002326965332 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:25.248 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-60c970ad-3883-4431-ba92-9c7647123620 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:25.270 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-60c970ad-3883-4431-ba92-9c7647123620 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:25.317 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-60c970ad-3883-4431-ba92-9c7647123620 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:25.379 4076 3051113294840 MainThread INFO nova.compute.manager [req-60c970ad-3883-4431-ba92-9c7647123620 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:25.520 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': '4C1F882C-98B8-433A-AD2D-6A45AF52E132', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14574), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:25.642 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:25.642 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:25.642 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:25.656 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] VM Stopped (Lifecycle Event) 2021-09-10 17:00:25.656 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job failed: {'RawErrors': (('124\'instance-00000044\' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF7812310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000044\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)instance-0000004483AC7C9D-2673-47A7-A349-B1291DF78123%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4C1F882C-98B8-433A-AD2D-6A45AF52E132', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112268), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\nNot enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)\n\n'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123)", 'ErrorSummaryDescription': "'instance-00000044' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000044.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:25.656 4076 3051113294136 MainThread ERROR nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Failed to change vm state of instance-00000044 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. 2021-09-10 17:00:25.656 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Got request to destroy instance 2021-09-10 17:00:25.687 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-5d4501bd-326f-406c-8e73-71bd0368c0fc - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:25.687 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-5d4501bd-326f-406c-8e73-71bd0368c0fc - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:25.711 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-5d4501bd-326f-406c-8e73-71bd0368c0fc - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:25.770 4076 3051113294840 MainThread INFO nova.compute.manager [req-5d4501bd-326f-406c-8e73-71bd0368c0fc - - - - -] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:26.288 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': '96BF281D-5D5C-48EF-92C2-BDBDCF823BF0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14183), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:26.397 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:26.397 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:26.397 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:26.397 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:26.420 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '96BF281D-5D5C-48EF-92C2-BDBDCF823BF0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=92020), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:26.420 4076 3051112930120 MainThread DEBUG os_win._utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3525097370147705 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:26.459 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-2b9297e7-bfd1-4c0e-bf2c-8dcffdf9db33 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:26.459 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-2b9297e7-bfd1-4c0e-bf2c-8dcffdf9db33 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:26.524 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-2b9297e7-bfd1-4c0e-bf2c-8dcffdf9db33 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:26.550 4076 3051113294840 MainThread INFO nova.compute.manager [req-2b9297e7-bfd1-4c0e-bf2c-8dcffdf9db33 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:26.943 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:26.966 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:00:27.071 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.084 4076 3051112928888 MainThread DEBUG nova.virt.hardware [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:00:27.084 4076 3051112928888 MainThread INFO nova.compute.claims [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Claim successful on node n-h1-808474-1 2021-09-10 17:00:27.196 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.hostops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:27.216 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.hostops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:27.332 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-81d39151-3297-4fb2-8b13-4d59def9c900 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:27.332 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-81d39151-3297-4fb2-8b13-4d59def9c900 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.332 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-81d39151-3297-4fb2-8b13-4d59def9c900 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.340 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-81d39151-3297-4fb2-8b13-4d59def9c900 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:27.340 4076 3051113294840 MainThread WARNING nova.compute.manager [req-81d39151-3297-4fb2-8b13-4d59def9c900 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-f630db82-46c7-4de7-a3e1-4ede08bcb598 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:27.426 4076 3051112928888 MainThread DEBUG nova.compute.provider_tree [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:27.443 4076 3051112928888 MainThread DEBUG nova.scheduler.client.report [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:27.443 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.443 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:00:27.474 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': 'A89D4398-63B3-437D-AE59-6666DA4498A7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=23559), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:27.525 4076 3051112928888 MainThread DEBUG nova.compute.utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:00:27.525 4076 3051113294840 GreenThread-57 DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:00:27.525 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:00:27.553 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:00:27.580 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:27.580 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.580 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.584 4076 3051112928712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:27.584 4076 3051112928712 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:27.584 4076 3051112929240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.584 4076 3051112929240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.584 4076 3051113294136 MainThread DEBUG os_win.utils.compute.vmutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed vm state of instance-00000044 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:27.584 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Successfully changed state of VM instance-00000044 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:27.614 4076 3051112928712 MainThread DEBUG nova.compute.manager [req-051e4e6a-4a8b-4504-a6cc-811483eb497b - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:27.614 4076 3051112928712 MainThread DEBUG nova.virt.hyperv.vmops [req-051e4e6a-4a8b-4504-a6cc-811483eb497b - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:27.614 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A89D4398-63B3-437D-AE59-6666DA4498A7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=96909), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:27.614 4076 3051112930120 MainThread DEBUG os_win._utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.546642303466797 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:27.661 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Waiting for WMI job: {'InstanceID': 'FA6A3285-3FD7-47F3-8253-DAF4F2117E6A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=26612), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:27.661 4076 3051112928712 MainThread DEBUG nova.compute.manager [req-051e4e6a-4a8b-4504-a6cc-811483eb497b - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:27.680 4076 3051112928712 MainThread INFO nova.compute.manager [req-051e4e6a-4a8b-4504-a6cc-811483eb497b - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:27.754 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:00:27.754 4076 3051112928888 MainThread INFO nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Spawning new instance 2021-09-10 17:00:27.770 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.770 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.770 4076 3051112928888 MainThread DEBUG nova.block_device [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:00:27.770 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.770 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.786 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] WMI job succeeded: {'InstanceID': 'FA6A3285-3FD7-47F3-8253-DAF4F2117E6A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29987), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:27.786 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.786 4076 3051113294136 MainThread DEBUG os_win.utils.pathutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Removing directory: C:\openstack\instances\instance-00000044 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:27.786 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.786 4076 3051113294136 MainThread ERROR nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Traceback (most recent call last): 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(*args, **kwargs) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] admin_password, network_info, block_device_info) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(self, *args, **kwds) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] should_plug_vifs=False) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'req_state': req_state}) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] time.sleep(sleep_time) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return f(*args, **kwargs) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return self._wait_for_job(job_path) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] error_desc=err_desc) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During handling of the above exception, another exception occurred: 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Traceback (most recent call last): 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] yield resources 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] accel_info=accel_info) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(*args, **kwargs) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] admin_password, network_info, block_device_info) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(self, *args, **kwds) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] should_plug_vifs=False) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'req_state': req_state}) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] time.sleep(sleep_time) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return f(*args, **kwargs) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return self._wait_for_job(job_path) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] error_desc=err_desc) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:27.786 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:27.786 4076 3051113294136 MainThread INFO nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Terminating instance 2021-09-10 17:00:27.786 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:27.800 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Got request to destroy instance 2021-09-10 17:00:27.800 4076 3051112928888 MainThread DEBUG os_win.utils.pathutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating directory: C:\openstack\instances\instance-00000046 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:00:27.800 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000046\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:00:27.816 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:27.816 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.816 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "instance-00000044" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:27.816 4076 3051113294136 MainThread INFO nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:27.816 4076 3051113294136 MainThread DEBUG nova.compute.claims [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:27.816 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:27.833 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Resizing VHD C:\openstack\instances\instance-00000046\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:00:27.926 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:27.954 4076 3051113161480 MainThread INFO nova.compute.manager [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] [instance: b0745679-4000-4aff-bd0a-29ded4711f24] Took 12.59 seconds to deallocate network for instance. 2021-09-10 17:00:27.954 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:27.974 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:28.101 4076 3051113294840 GreenThread-57 DEBUG nova.policy [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a1711a7b02c2433dacfb4017cd0a812c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e0b9dfa46b614e449b6aff88db2cfd6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:00:28.210 4076 3051113161480 MainThread INFO nova.scheduler.client.report [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Deleted allocations for instance b0745679-4000-4aff-bd0a-29ded4711f24 2021-09-10 17:00:28.223 4076 3051113294136 MainThread DEBUG nova.compute.provider_tree [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:28.223 4076 3051113294136 MainThread DEBUG nova.scheduler.client.report [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:28.223 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:28.255 4076 3051113294136 MainThread ERROR nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Traceback (most recent call last): 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(*args, **kwargs) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] admin_password, network_info, block_device_info) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(self, *args, **kwds) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] should_plug_vifs=False) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'req_state': req_state}) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] time.sleep(sleep_time) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return f(*args, **kwargs) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return self._wait_for_job(job_path) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] error_desc=err_desc) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] During handling of the above exception, another exception occurred: 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Traceback (most recent call last): 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] accel_info=accel_info) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(*args, **kwargs) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] admin_password, network_info, block_device_info) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return function(self, *args, **kwds) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] should_plug_vifs=False) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'req_state': req_state}) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] time.sleep(sleep_time) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self.force_reraise() 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] raise self.value 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return f(*args, **kwargs) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] return self._wait_for_job(job_path) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] error_desc=err_desc) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. 2021-09-10 17:00:28.255 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.compute.utils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Build of instance 4cd8ee20-649d-4e14-8c26-085e78ef091d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000044' failed to start. Not enough memory in the system to start the virtual machine instance-00000044. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000044' failed to start. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Not enough memory in the system to start the virtual machine instance-00000044 with ram size 1024 megabytes. (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) 'instance-00000044' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 83AC7C9D-2673-47A7-A349-B1291DF78123) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:28.255 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:28.285 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-89e3c9f1-cf51-4a41-8c00-ba192222310f e257d97e390840179aaf42ec0308f2de f00087122c344aa8a15842d2b03f9a16 - default default] Lock "b0745679-4000-4aff-bd0a-29ded4711f24" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 126.656s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:28.646 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': 'BD512709-A028-4EFB-A0EB-3B01AE48BE95', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15112), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:28.766 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:28.766 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:28.766 4076 3051112930296 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:28.766 4076 3051112930296 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:28.799 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-055e7326-db99-47d4-a17a-3f96a18e5475 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:28.799 4076 3051112930296 MainThread DEBUG nova.virt.hyperv.vmops [req-055e7326-db99-47d4-a17a-3f96a18e5475 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:28.799 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BD512709-A028-4EFB-A0EB-3B01AE48BE95', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=89346), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:28.799 4076 3051112930120 MainThread DEBUG os_win._utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.731194257736206 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:28.846 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-055e7326-db99-47d4-a17a-3f96a18e5475 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:28.879 4076 3051112930296 MainThread INFO nova.compute.manager [req-055e7326-db99-47d4-a17a-3f96a18e5475 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:29.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:29.832 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': '7942DF3E-3D31-4513-837E-36AD56406C5C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15070), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:29.950 4076 3051112930296 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:29.950 4076 3051112930296 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:29.950 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:29.950 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:29.974 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-7131c361-caad-40e3-98b7-9fd7f2362d9d - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:29.974 4076 3051112930296 MainThread DEBUG nova.virt.hyperv.vmops [req-7131c361-caad-40e3-98b7-9fd7f2362d9d - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:29.974 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7942DF3E-3D31-4513-837E-36AD56406C5C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=94781), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:29.974 4076 3051112930120 MainThread DEBUG os_win._utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.906158924102783 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:30.015 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-7131c361-caad-40e3-98b7-9fd7f2362d9d - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:30.094 4076 3051112930296 MainThread INFO nova.compute.manager [req-7131c361-caad-40e3-98b7-9fd7f2362d9d - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:30.318 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.318 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.318 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.318 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.318 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-9f4f550b-e4d6-4c6b-9ae8-78b11fd5034c for instance with vm_state building and task_state spawning. 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.333 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-db6ca41c-54e9-4053-bd66-ca4521a281c9 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-add97417-8a27-4260-8deb-932d14156785 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-add97417-8a27-4260-8deb-932d14156785 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.333 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-add97417-8a27-4260-8deb-932d14156785 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.333 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-1f3d3575-c85f-4f80-88aa-29e1aeca2b5a for instance with vm_state building and task_state spawning. 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-ffc6e67e-0a49-4c79-9748-12da98323372 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-ffc6e67e-0a49-4c79-9748-12da98323372 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.333 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-ffc6e67e-0a49-4c79-9748-12da98323372 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received event network-vif-unplugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG oslo_concurrency.lockutils [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:30.333 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] No waiting events found dispatching network-vif-unplugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:00:30.333 4076 3051112930296 MainThread WARNING nova.compute.manager [req-9e3123a1-3e98-43d0-b9bb-727cc1772393 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Received unexpected event network-vif-unplugged-2328bac5-09f9-4dd3-a3ff-1f5673edded4 for instance with vm_state building and task_state spawning. 2021-09-10 17:00:31.005 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': '6122837F-E5F7-4F58-9844-33905653D71B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15021), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:31.127 4076 3051112930296 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:31.127 4076 3051112930296 MainThread INFO nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] VM Stopped (Lifecycle Event) 2021-09-10 17:00:31.127 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:31.127 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:31.146 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-d0b83e0d-3ff4-4296-9e1c-fff2c21ac2b8 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:31.146 4076 3051112930296 MainThread DEBUG nova.virt.hyperv.vmops [req-d0b83e0d-3ff4-4296-9e1c-fff2c21ac2b8 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:31.146 4076 3051112930120 MainThread ERROR os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000045\' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F5710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000045\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)instance-00000045359938C0-F804-4263-AD9C-FA1C32642F57%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6122837F-E5F7-4F58-9844-33905653D71B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=91333), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\nNot enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)\n\n'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57)", 'ErrorSummaryDescription': "'instance-00000045' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000045.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:31.146 4076 3051112930120 MainThread ERROR nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Failed to change vm state of instance-00000045 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. 2021-09-10 17:00:31.146 4076 3051112930120 MainThread INFO nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Got request to destroy instance 2021-09-10 17:00:31.233 4076 3051112930296 MainThread DEBUG nova.compute.manager [req-d0b83e0d-3ff4-4296-9e1c-fff2c21ac2b8 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:31.271 4076 3051112930296 MainThread INFO nova.compute.manager [req-d0b83e0d-3ff4-4296-9e1c-fff2c21ac2b8 - - - - -] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:31.309 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:31.340 4076 3051113294136 MainThread INFO nova.compute.manager [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] [instance: 4cd8ee20-649d-4e14-8c26-085e78ef091d] Took 3.08 seconds to deallocate network for instance. 2021-09-10 17:00:31.493 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Successfully created port: c28da0b6-d5f9-4629-b9db-9907a00d8173 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:00:31.620 4076 3051113294136 MainThread INFO nova.scheduler.client.report [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Deleted allocations for instance 4cd8ee20-649d-4e14-8c26-085e78ef091d 2021-09-10 17:00:31.680 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-78cb4793-49c6-427e-973e-ae43812fe37c 42267190de3648eea52604e6b2444d20 6ca07d9dc99a4581a698a232ffa065db - default default] Lock "4cd8ee20-649d-4e14-8c26-085e78ef091d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:32.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:32.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:32.786 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:00:32.923 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:32.923 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:32.923 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:32.942 4076 3051112930120 MainThread DEBUG os_win.utils.compute.vmutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Successfully changed vm state of instance-00000045 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:32.942 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Successfully changed state of VM instance-00000045 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:33.000 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for WMI job: {'InstanceID': '5BBDFEE5-82D3-419A-9B01-D57DA28D7D51', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14919), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG os_win.utils.jobutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] WMI job succeeded: {'InstanceID': '5BBDFEE5-82D3-419A-9B01-D57DA28D7D51', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=27346), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG os_win.utils.pathutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Removing directory: C:\openstack\instances\instance-00000045 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:33.114 4076 3051112930120 MainThread ERROR nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Traceback (most recent call last): 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(*args, **kwargs) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] admin_password, network_info, block_device_info) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(self, *args, **kwds) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] should_plug_vifs=False) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'req_state': req_state}) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] time.sleep(sleep_time) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return f(*args, **kwargs) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return self._wait_for_job(job_path) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] error_desc=err_desc) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During handling of the above exception, another exception occurred: 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Traceback (most recent call last): 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] yield resources 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] accel_info=accel_info) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(*args, **kwargs) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] admin_password, network_info, block_device_info) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(self, *args, **kwds) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] should_plug_vifs=False) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'req_state': req_state}) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] time.sleep(sleep_time) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return f(*args, **kwargs) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return self._wait_for_job(job_path) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] error_desc=err_desc) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.114 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.114 4076 3051112930120 MainThread INFO nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Terminating instance 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Acquired lock "refresh_cache-6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:33.114 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:00:33.176 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:33.176 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:00:33.212 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:00:33.431 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:33.474 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Releasing lock "refresh_cache-6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:33.474 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:33.474 4076 3051112930120 MainThread INFO nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Got request to destroy instance 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "instance-00000045" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:33.492 4076 3051112930120 MainThread INFO nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG oslo.service.loopingcall [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.._deallocate_network_with_retries to return. func c:\python37\lib\site-packages\oslo_service\loopingcall.py:435 2021-09-10 17:00:33.492 4076 3051113294136 GreenThread-58 DEBUG nova.compute.manager [-] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG nova.compute.claims [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:33.492 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:33.603 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.hostops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:33.629 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.hostops [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:33.836 4076 3051112930120 MainThread DEBUG nova.compute.provider_tree [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:33.848 4076 3051112930120 MainThread DEBUG nova.scheduler.client.report [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:33.848 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:33.848 4076 3051112930120 MainThread ERROR nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Traceback (most recent call last): 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(*args, **kwargs) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] admin_password, network_info, block_device_info) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(self, *args, **kwds) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] should_plug_vifs=False) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'req_state': req_state}) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] time.sleep(sleep_time) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return f(*args, **kwargs) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return self._wait_for_job(job_path) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] error_desc=err_desc) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] During handling of the above exception, another exception occurred: 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Traceback (most recent call last): 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] accel_info=accel_info) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(*args, **kwargs) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] admin_password, network_info, block_device_info) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return function(self, *args, **kwds) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] should_plug_vifs=False) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'req_state': req_state}) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] time.sleep(sleep_time) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self.force_reraise() 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] raise self.value 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return f(*args, **kwargs) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] return self._wait_for_job(job_path) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] error_desc=err_desc) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. 2021-09-10 17:00:33.848 4076 3051112930120 MainThread TRACE nova.compute.manager [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] 2021-09-10 17:00:33.848 4076 3051112930120 MainThread DEBUG nova.compute.utils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:33.848 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Build of instance 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000045' failed to start. Not enough memory in the system to start the virtual machine instance-00000045. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000045' failed to start. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Not enough memory in the system to start the virtual machine instance-00000045 with ram size 1024 megabytes. (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) 'instance-00000045' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 359938C0-F804-4263-AD9C-FA1C32642F57) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:33.864 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:33.864 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Acquired lock "refresh_cache-6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:33.864 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:00:33.926 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:00:34.457 4076 3051112930120 MainThread DEBUG nova.network.neutron [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:34.476 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Releasing lock "refresh_cache-6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:34.476 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:34.476 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] [instance: 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:00:34.707 4076 3051112930120 MainThread INFO nova.scheduler.client.report [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Deleted allocations for instance 6d2c0c79-952c-46e8-9d1f-e8c9891d61d5 2021-09-10 17:00:34.748 4076 3051112930120 MainThread DEBUG oslo_concurrency.lockutils [req-36b84dc7-537c-4ab8-b96d-3162388faeca b650955abba04294b2c6b5e7f0734423 69b5dbd9fc104f8aa8e9386ba8229ec1 - default default] Lock "6d2c0c79-952c-46e8-9d1f-e8c9891d61d5" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 13.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:35.068 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Successfully updated port: c28da0b6-d5f9-4629-b9db-9907a00d8173 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:00:35.098 4076 3051113294840 GreenThread-57 DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Acquired lock "refresh_cache-d5135b07-1b25-46d3-8c64-016de11558e4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:35.098 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:00:35.548 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:00:35.880 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:35.911 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:35.911 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:00:35.911 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:35.911 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:36.192 4076 3051049642296 GreenThread-2 INFO nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running instance usage audit for host n-h1-808474-1 from 2021-09-10 16:00:00 to 2021-09-10 17:00:00. 12 instances. 2021-09-10 17:00:36.239 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:36.796 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:36.858 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:36.876 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:36.876 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:36.876 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:00:36.876 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:36.896 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:36.896 4076 3051113294840 GreenThread-57 DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Updating instance_info_cache with network_info: [{"id": "c28da0b6-d5f9-4629-b9db-9907a00d8173", "address": "fa:16:3e:66:61:93", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc28da0b6-d5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:36.929 4076 3051113294840 GreenThread-57 DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Releasing lock "refresh_cache-d5135b07-1b25-46d3-8c64-016de11558e4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:36.929 4076 3051113294840 GreenThread-57 DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Instance network_info: |[{"id": "c28da0b6-d5f9-4629-b9db-9907a00d8173", "address": "fa:16:3e:66:61:93", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc28da0b6-d5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:00:36.929 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Preparing to wait for external event network-vif-plugged-c28da0b6-d5f9-4629-b9db-9907a00d8173 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:00:36.929 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:36.929 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:36.929 4076 3051112928888 MainThread DEBUG os_win.utils.compute.vmutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Creating VM instance-00000046 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:00:37.116 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2173MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:00:37.116 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.276 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.291 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance d5135b07-1b25-46d3-8c64-016de11558e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:00:37.291 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 14 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:00:37.291 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=14848MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=14 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:00:37.333 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:37.349 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:37.562 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:37.578 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:37.578 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:00:37.578 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:37.817 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:00:38.207 4076 3051113162008 MainThread INFO nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Took 16.14 seconds to detach 3 volumes for instance. 2021-09-10 17:00:38.209 4076 3051113162008 MainThread DEBUG nova.compute.claims [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:38.209 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:38.418 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.hostops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:38.432 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.hostops [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:38.520 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:38.629 4076 3051113162008 MainThread DEBUG nova.compute.provider_tree [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.scheduler.client.report [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:38.660 4076 3051113162008 MainThread ERROR nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Traceback (most recent call last): 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(*args, **kwargs) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] admin_password, network_info, block_device_info) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(self, *args, **kwds) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] should_plug_vifs=False) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'req_state': req_state}) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] time.sleep(sleep_time) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return f(*args, **kwargs) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return self._wait_for_job(job_path) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] error_desc=err_desc) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] During handling of the above exception, another exception occurred: 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Traceback (most recent call last): 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] accel_info=accel_info) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(*args, **kwargs) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] admin_password, network_info, block_device_info) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return function(self, *args, **kwds) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] should_plug_vifs=False) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'req_state': req_state}) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] time.sleep(sleep_time) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self.force_reraise() 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] raise self.value 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return f(*args, **kwargs) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] return self._wait_for_job(job_path) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] error_desc=err_desc) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. 2021-09-10 17:00:38.660 4076 3051113162008 MainThread TRACE nova.compute.manager [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.compute.utils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Build of instance d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000003f' failed to start. Not enough memory in the system to start the virtual machine instance-0000003f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000003f' failed to start. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Not enough memory in the system to start the virtual machine instance-0000003f with ram size 1024 megabytes. (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) 'instance-0000003f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID AF4B0BD4-B5B2-4F3F-BAA7-EDB2488F9B1E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:38.660 4076 3051113162008 MainThread DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:39.793 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:00:39.793 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:00:39.793 4076 3051112929944 MainThread DEBUG nova.compute.manager [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Received event network-changed-c28da0b6-d5f9-4629-b9db-9907a00d8173 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:39.793 4076 3051112929944 MainThread DEBUG nova.compute.manager [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Refreshing instance network info cache due to event network-changed-c28da0b6-d5f9-4629-b9db-9907a00d8173. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:00:39.793 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d5135b07-1b25-46d3-8c64-016de11558e4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:00:39.793 4076 3051112929944 MainThread DEBUG nova.network.neutron [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Refreshing network info cache for port c28da0b6-d5f9-4629-b9db-9907a00d8173 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:00:42.458 4076 3051112929944 MainThread DEBUG nova.network.neutron [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Updated VIF entry in instance network info cache for port c28da0b6-d5f9-4629-b9db-9907a00d8173. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:00:42.458 4076 3051112929944 MainThread DEBUG nova.network.neutron [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Updating instance_info_cache with network_info: [{"id": "c28da0b6-d5f9-4629-b9db-9907a00d8173", "address": "fa:16:3e:66:61:93", "network": {"id": "3eb3df69-b47c-4acb-8098-7e28b146140d", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-1037400376-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e0b9dfa46b614e449b6aff88db2cfd6b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc28da0b6-d5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:42.482 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [req-c2c56015-ac3e-4d08-b702-a529e34fa87b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d5135b07-1b25-46d3-8c64-016de11558e4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:00:43.188 4076 3051112929944 MainThread DEBUG nova.compute.manager [req-ae0bef88-933a-492c-91de-fc7070e6137b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Received event network-vif-plugged-c28da0b6-d5f9-4629-b9db-9907a00d8173 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:00:43.188 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [req-ae0bef88-933a-492c-91de-fc7070e6137b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:43.188 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [req-ae0bef88-933a-492c-91de-fc7070e6137b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:43.188 4076 3051112929944 MainThread DEBUG nova.compute.manager [req-ae0bef88-933a-492c-91de-fc7070e6137b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Processing event network-vif-plugged-c28da0b6-d5f9-4629-b9db-9907a00d8173 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:00:43.224 4076 3051112928888 MainThread INFO nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Using config drive for instance 2021-09-10 17:00:44.175 4076 3051112928888 MainThread INFO nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Creating config drive at C:\openstack\instances\instance-00000046\configdrive.iso 2021-09-10 17:00:44.220 4076 3051112928888 MainThread DEBUG oslo_concurrency.processutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000046\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps39x1iju execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:00:44.531 4076 3051112928888 MainThread DEBUG oslo_concurrency.processutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000046\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps39x1iju" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:00:44.915 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Setting boot order for instance: instance-00000046: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:00:44.959 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:00:44.982 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'BE9B6464-7C78-40D9-A0F5-F913FDF4A1C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14888), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:45.136 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:45.136 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:45.136 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:45.136 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:45.136 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BE9B6464-7C78-40D9-A0F5-F913FDF4A1C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112314), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:45.136 4076 3051112928888 MainThread DEBUG os_win._utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17673730850219727 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:45.156 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-0597b33a-ad84-4d94-93a3-73ba7dd3776f - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:45.156 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-0597b33a-ad84-4d94-93a3-73ba7dd3776f - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:45.208 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-0597b33a-ad84-4d94-93a3-73ba7dd3776f - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:45.228 4076 3051112930120 MainThread INFO nova.compute.manager [req-0597b33a-ad84-4d94-93a3-73ba7dd3776f - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:46.178 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '667F5B75-B93C-4EAB-9DE4-8BB6ED1D6E36', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14406), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:46.301 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:46.301 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:46.301 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:46.301 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:46.339 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '667F5B75-B93C-4EAB-9DE4-8BB6ED1D6E36', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109936), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:46.339 4076 3051112928888 MainThread DEBUG os_win._utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3799426555633545 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:46.339 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-e8963ad2-59e8-45df-89dd-46b488b74168 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:46.339 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-e8963ad2-59e8-45df-89dd-46b488b74168 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:46.386 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-e8963ad2-59e8-45df-89dd-46b488b74168 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:46.425 4076 3051112930120 MainThread INFO nova.compute.manager [req-e8963ad2-59e8-45df-89dd-46b488b74168 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:47.379 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': 'A9A06A0B-4350-499D-96EF-CD56F3016F31', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15252), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:47.489 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:47.513 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:47.513 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:47.513 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:47.513 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A9A06A0B-4350-499D-96EF-CD56F3016F31', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109448), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:47.513 4076 3051112928888 MainThread DEBUG os_win._utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5537164211273193 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:47.513 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-026fea3e-0164-4965-b65c-074f61e17fad - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:47.513 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-026fea3e-0164-4965-b65c-074f61e17fad - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:47.570 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-026fea3e-0164-4965-b65c-074f61e17fad - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:47.591 4076 3051112930120 MainThread INFO nova.compute.manager [req-026fea3e-0164-4965-b65c-074f61e17fad - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:48.539 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '8365ADCA-528B-4D56-BAF9-142445A1B5A6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14454), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:48.662 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:48.662 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:48.662 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:48.662 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:48.685 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8365ADCA-528B-4D56-BAF9-142445A1B5A6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109962), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:48.685 4076 3051112928888 MainThread DEBUG os_win._utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7258048057556152 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:48.685 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-e0bb9ee1-c2ff-4350-aa9c-6987773312c9 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:48.685 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-e0bb9ee1-c2ff-4350-aa9c-6987773312c9 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:48.740 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-e0bb9ee1-c2ff-4350-aa9c-6987773312c9 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:48.766 4076 3051112930120 MainThread INFO nova.compute.manager [req-e0bb9ee1-c2ff-4350-aa9c-6987773312c9 - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:49.723 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '9618EC2D-6B6E-4178-B9C8-69CC4F10C118', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15181), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:49.847 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:49.847 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:49.847 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:49.847 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:49.862 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-8ca0c52e-7aef-4625-ac14-037a3c63e0dd - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:49.862 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-8ca0c52e-7aef-4625-ac14-037a3c63e0dd - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:49.884 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9618EC2D-6B6E-4178-B9C8-69CC4F10C118', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111217), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:49.884 4076 3051112928888 MainThread DEBUG os_win._utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.925051689147949 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:00:49.917 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-8ca0c52e-7aef-4625-ac14-037a3c63e0dd - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:49.947 4076 3051112930120 MainThread INFO nova.compute.manager [req-8ca0c52e-7aef-4625-ac14-037a3c63e0dd - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:50.910 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '111CFBAF-D115-487C-BCCA-CDB4D606F264', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14743), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:51.048 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:51.048 4076 3051112929944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:51.048 4076 3051112930120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:00:51.048 4076 3051112930120 MainThread INFO nova.compute.manager [-] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] VM Stopped (Lifecycle Event) 2021-09-10 17:00:51.048 4076 3051112928888 MainThread ERROR os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000046\' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE69610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000046\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)instance-00000046361C8028-D278-4BB9-A720-30519F0EE696%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '111CFBAF-D115-487C-BCCA-CDB4D606F264', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112616), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 0, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 0, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\nNot enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)\n\n'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696)", 'ErrorSummaryDescription': "'instance-00000046' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000046.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:00:51.058 4076 3051112928888 MainThread ERROR nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Failed to change vm state of instance-00000046 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. 2021-09-10 17:00:51.058 4076 3051112928888 MainThread INFO nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Got request to destroy instance 2021-09-10 17:00:51.071 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-1bf5b6ac-daef-4629-bcfa-73e78d32beef - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:00:51.071 4076 3051112930120 MainThread DEBUG nova.virt.hyperv.vmops [req-1bf5b6ac-daef-4629-bcfa-73e78d32beef - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:00:51.118 4076 3051112930120 MainThread DEBUG nova.compute.manager [req-1bf5b6ac-daef-4629-bcfa-73e78d32beef - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:00:51.136 4076 3051112930120 MainThread INFO nova.compute.manager [req-1bf5b6ac-daef-4629-bcfa-73e78d32beef - - - - -] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:00:52.229 4076 3051113162008 MainThread DEBUG nova.network.neutron [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:52.286 4076 3051113162008 MainThread INFO nova.compute.manager [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] [instance: d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869] Took 13.62 seconds to deallocate network for instance. 2021-09-10 17:00:52.598 4076 3051113162008 MainThread INFO nova.scheduler.client.report [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Deleted allocations for instance d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869 2021-09-10 17:00:52.631 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [req-6d233a42-e81a-4392-ab3d-5e95c272f207 b736354cedde4200845a6c18d167eda3 fe5b53ea59bd43bf84431d6daec45a04 - default default] Lock "d4b1aa5b-c1ae-4df7-afd7-15d31bbaa869" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 103.140s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:52.910 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:00:52.913 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:52.913 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:52.927 4076 3051112928888 MainThread DEBUG os_win.utils.compute.vmutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed vm state of instance-00000046 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:00:52.927 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Successfully changed state of VM instance-00000046 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:00:52.973 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Waiting for WMI job: {'InstanceID': '9FAE23A4-029B-4689-AC99-D69482DB7E55', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14790), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:00:53.098 4076 3051112928888 MainThread DEBUG os_win.utils.jobutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] WMI job succeeded: {'InstanceID': '9FAE23A4-029B-4689-AC99-D69482DB7E55', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29417), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG os_win.utils.pathutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Removing directory: C:\openstack\instances\instance-00000046 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:53.100 4076 3051112928888 MainThread ERROR nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Traceback (most recent call last): 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(*args, **kwargs) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] admin_password, network_info, block_device_info) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(self, *args, **kwds) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] should_plug_vifs=False) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'req_state': req_state}) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] time.sleep(sleep_time) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return f(*args, **kwargs) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return self._wait_for_job(job_path) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] error_desc=err_desc) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During handling of the above exception, another exception occurred: 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Traceback (most recent call last): 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] yield resources 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] accel_info=accel_info) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(*args, **kwargs) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] admin_password, network_info, block_device_info) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(self, *args, **kwds) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] should_plug_vifs=False) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'req_state': req_state}) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] time.sleep(sleep_time) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return f(*args, **kwargs) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return self._wait_for_job(job_path) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] error_desc=err_desc) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.100 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.100 4076 3051112928888 MainThread INFO nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Terminating instance 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:00:53.100 4076 3051112928888 MainThread INFO nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Got request to destroy instance 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:00:53.100 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:53.127 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "instance-00000046" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:53.127 4076 3051112928888 MainThread INFO nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:00:53.127 4076 3051112928888 MainThread DEBUG nova.compute.claims [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:00:53.127 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:00:53.332 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.hostops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:00:53.348 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.hostops [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:00:53.561 4076 3051112928888 MainThread DEBUG nova.compute.provider_tree [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.scheduler.client.report [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:00:53.566 4076 3051112928888 MainThread ERROR nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Traceback (most recent call last): 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(*args, **kwargs) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] admin_password, network_info, block_device_info) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(self, *args, **kwds) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] should_plug_vifs=False) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'req_state': req_state}) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] time.sleep(sleep_time) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return f(*args, **kwargs) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return self._wait_for_job(job_path) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] error_desc=err_desc) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] During handling of the above exception, another exception occurred: 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Traceback (most recent call last): 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] accel_info=accel_info) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(*args, **kwargs) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] admin_password, network_info, block_device_info) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return function(self, *args, **kwds) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] should_plug_vifs=False) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'req_state': req_state}) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] time.sleep(sleep_time) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self.force_reraise() 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] raise self.value 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return f(*args, **kwargs) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] return self._wait_for_job(job_path) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] error_desc=err_desc) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. 2021-09-10 17:00:53.566 4076 3051112928888 MainThread TRACE nova.compute.manager [instance: d5135b07-1b25-46d3-8c64-016de11558e4] 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.compute.utils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Build of instance d5135b07-1b25-46d3-8c64-016de11558e4 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000046' failed to start. Not enough memory in the system to start the virtual machine instance-00000046. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000046' failed to start. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Not enough memory in the system to start the virtual machine instance-00000046 with ram size 1024 megabytes. (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) 'instance-00000046' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 361C8028-D278-4BB9-A720-30519F0EE696) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:00:53.566 4076 3051112928888 MainThread DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:00:55.919 4076 3051112928888 MainThread DEBUG nova.network.neutron [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:00:55.948 4076 3051112928888 MainThread INFO nova.compute.manager [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] [instance: d5135b07-1b25-46d3-8c64-016de11558e4] Took 2.38 seconds to deallocate network for instance. 2021-09-10 17:00:56.251 4076 3051112928888 MainThread INFO nova.scheduler.client.report [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Deleted allocations for instance d5135b07-1b25-46d3-8c64-016de11558e4 2021-09-10 17:00:56.275 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [req-b42a718c-dde8-46a1-97e9-77fbee6ea660 a1711a7b02c2433dacfb4017cd0a812c e0b9dfa46b614e449b6aff88db2cfd6b - default default] Lock "d5135b07-1b25-46d3-8c64-016de11558e4" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:00.254 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:00.287 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:00.445 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:00.445 4076 3051113294840 MainThread DEBUG nova.virt.hardware [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:00.445 4076 3051113294840 MainThread INFO nova.compute.claims [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Claim successful on node n-h1-808474-1 2021-09-10 17:01:00.646 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:00.660 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:00.875 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:00.891 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:00.891 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:00.891 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:00.973 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:00.973 4076 3051112928888 GreenThread-59 DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:00.973 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:01.009 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:01.171 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:01.197 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Spawning new instance 2021-09-10 17:01:01.197 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:01.197 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:01.197 4076 3051113294840 MainThread DEBUG nova.block_device [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:01.197 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:01.197 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:01.214 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Creating directory: C:\openstack\instances\instance-00000047 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:01.214 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000047\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:01.239 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Resizing VHD C:\openstack\instances\instance-00000047\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:01.394 4076 3051112928888 GreenThread-59 DEBUG nova.policy [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3f85143bbedf44eca37aba098b6ccb7d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f2d0543bdd24962be5e0dceb47ee53e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:04.131 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Successfully created port: 63104a25-8d60-4319-8d76-624a157b5f14 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:07.291 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:07.332 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:07.488 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:07.488 4076 3051095191800 MainThread DEBUG nova.virt.hardware [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:07.488 4076 3051095191800 MainThread INFO nova.compute.claims [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Claim successful on node n-h1-808474-1 2021-09-10 17:01:07.663 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.hostops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:07.676 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.hostops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:07.883 4076 3051095191800 MainThread DEBUG nova.compute.provider_tree [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:07.896 4076 3051095191800 MainThread DEBUG nova.scheduler.client.report [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:07.896 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:07.896 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:07.978 4076 3051095191800 MainThread DEBUG nova.compute.utils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:07.978 4076 3051095191624 GreenThread-60 DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:07.978 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:08.021 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:08.247 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Successfully updated port: 63104a25-8d60-4319-8d76-624a157b5f14 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:08.270 4076 3051112928888 GreenThread-59 DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Acquired lock "refresh_cache-219ee2c7-854a-4265-903f-5d10381fa708" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:08.270 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:08.644 4076 3051095191624 GreenThread-60 DEBUG nova.policy [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:08.660 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:09.318 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Received event network-changed-63104a25-8d60-4319-8d76-624a157b5f14 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:09.318 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Refreshing instance network info cache due to event network-changed-63104a25-8d60-4319-8d76-624a157b5f14. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:10.523 4076 3051112928888 GreenThread-59 DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Updating instance_info_cache with network_info: [{"id": "63104a25-8d60-4319-8d76-624a157b5f14", "address": "fa:16:3e:48:77:e5", "network": {"id": "6383d5d0-f1bd-4671-89b0-0e4e06a1b5c2", "bridge": null, "label": "tempest-ServerPasswordTestJSON-1310888354-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5f2d0543bdd24962be5e0dceb47ee53e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap63104a25-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:10.582 4076 3051112928888 GreenThread-59 DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Releasing lock "refresh_cache-219ee2c7-854a-4265-903f-5d10381fa708" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:10.582 4076 3051112928888 GreenThread-59 DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Instance network_info: |[{"id": "63104a25-8d60-4319-8d76-624a157b5f14", "address": "fa:16:3e:48:77:e5", "network": {"id": "6383d5d0-f1bd-4671-89b0-0e4e06a1b5c2", "bridge": null, "label": "tempest-ServerPasswordTestJSON-1310888354-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5f2d0543bdd24962be5e0dceb47ee53e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap63104a25-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:10.582 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Preparing to wait for external event network-vif-plugged-63104a25-8d60-4319-8d76-624a157b5f14 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:10.582 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:10.582 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:10.582 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Creating VM instance-00000047 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:10.582 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-219ee2c7-854a-4265-903f-5d10381fa708" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:10.582 4076 3051095192152 MainThread DEBUG nova.network.neutron [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Refreshing network info cache for port 63104a25-8d60-4319-8d76-624a157b5f14 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:11.443 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:12.053 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Successfully created port: 4b0bfd3a-5b69-494a-9940-79c70157a96d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:13.069 4076 3051095192152 MainThread DEBUG nova.network.neutron [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Updated VIF entry in instance network info cache for port 63104a25-8d60-4319-8d76-624a157b5f14. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:13.069 4076 3051095192152 MainThread DEBUG nova.network.neutron [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Updating instance_info_cache with network_info: [{"id": "63104a25-8d60-4319-8d76-624a157b5f14", "address": "fa:16:3e:48:77:e5", "network": {"id": "6383d5d0-f1bd-4671-89b0-0e4e06a1b5c2", "bridge": null, "label": "tempest-ServerPasswordTestJSON-1310888354-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5f2d0543bdd24962be5e0dceb47ee53e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap63104a25-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:13.094 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-87e32ad4-f438-43c9-8c83-194a8bf689a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-219ee2c7-854a-4265-903f-5d10381fa708" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:14.570 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:14.598 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:14.693 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:14.693 4076 3051095192152 MainThread DEBUG nova.virt.hardware [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:14.693 4076 3051095192152 MainThread INFO nova.compute.claims [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Claim successful on node n-h1-808474-1 2021-09-10 17:01:14.786 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.hostops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:14.821 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.hostops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:15.037 4076 3051095192152 MainThread DEBUG nova.compute.provider_tree [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:15.037 4076 3051095192152 MainThread DEBUG nova.scheduler.client.report [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:15.037 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:15.037 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:15.114 4076 3051095192152 MainThread DEBUG nova.compute.utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:15.114 4076 3051112928888 GreenThread-61 DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:15.114 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:15.137 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:15.254 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:15.281 4076 3051095192152 MainThread INFO nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Spawning new instance 2021-09-10 17:01:15.281 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Successfully updated port: 4b0bfd3a-5b69-494a-9940-79c70157a96d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:15.297 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:15.297 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:15.297 4076 3051095192152 MainThread DEBUG nova.block_device [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:15.297 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:15.297 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:15.305 4076 3051095192152 MainThread DEBUG os_win.utils.pathutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Creating directory: C:\openstack\instances\instance-0000004a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:15.305 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000004a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:15.305 4076 3051095191624 GreenThread-60 DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-9147ea0e-c0cc-436f-802f-3d7cbccd9529" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:15.305 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:15.332 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Resizing VHD C:\openstack\instances\instance-0000004a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:15.519 4076 3051112928888 GreenThread-61 DEBUG nova.policy [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c5eaa901834b4ee885c2b6a9ce410303', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0c42eab1e5884dc19398c04cdcea9edd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:15.542 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:16.394 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Received event network-changed-4b0bfd3a-5b69-494a-9940-79c70157a96d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:16.410 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Refreshing instance network info cache due to event network-changed-4b0bfd3a-5b69-494a-9940-79c70157a96d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:16.816 4076 3051095191624 GreenThread-60 DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Updating instance_info_cache with network_info: [{"id": "4b0bfd3a-5b69-494a-9940-79c70157a96d", "address": "fa:16:3e:c3:ce:26", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4b0bfd3a-5b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:16.832 4076 3051095191624 GreenThread-60 DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-9147ea0e-c0cc-436f-802f-3d7cbccd9529" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:16.832 4076 3051095191624 GreenThread-60 DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Instance network_info: |[{"id": "4b0bfd3a-5b69-494a-9940-79c70157a96d", "address": "fa:16:3e:c3:ce:26", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4b0bfd3a-5b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:16.832 4076 3051095191800 MainThread INFO nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Terminating instance 2021-09-10 17:01:16.832 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:16.832 4076 3051095191800 MainThread INFO nova.virt.hyperv.vmops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Got request to destroy instance 2021-09-10 17:01:16.832 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9147ea0e-c0cc-436f-802f-3d7cbccd9529" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:16.832 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Refreshing network info cache for port 4b0bfd3a-5b69-494a-9940-79c70157a96d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:16.848 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.vmops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:16.848 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000049" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:16.848 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000049" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:16.848 4076 3051095191800 MainThread INFO nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:16.848 4076 3051095191800 MainThread DEBUG nova.compute.claims [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:16.848 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:16.966 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.hostops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:16.981 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.hostops [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:17.182 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-58b5ad02-8c69-4c62-83bf-c7393dc80ee3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Received event network-vif-plugged-63104a25-8d60-4319-8d76-624a157b5f14 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:17.182 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-58b5ad02-8c69-4c62-83bf-c7393dc80ee3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:17.182 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-58b5ad02-8c69-4c62-83bf-c7393dc80ee3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:17.182 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-58b5ad02-8c69-4c62-83bf-c7393dc80ee3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Processing event network-vif-plugged-63104a25-8d60-4319-8d76-624a157b5f14 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:01:17.182 4076 3051095191800 MainThread DEBUG nova.compute.provider_tree [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:17.238 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Using config drive for instance 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.scheduler.client.report [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.compute.utils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Conflict updating instance 9147ea0e-c0cc-436f-802f-3d7cbccd9529. Expected: {'task_state': ['block_device_mapping']}. Actual: {'task_state': 'deleting'} notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Instance disappeared during build. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2272 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:17.238 4076 3051095191800 MainThread DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:01:17.604 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Creating config drive at C:\openstack\instances\instance-00000047\configdrive.iso 2021-09-10 17:01:17.630 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000047\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpgf1m3j0z execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:01:17.962 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000047\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpgf1m3j0z" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:01:18.083 4076 3051095191800 MainThread DEBUG nova.network.neutron [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:18.134 4076 3051095191800 MainThread INFO nova.compute.manager [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Took 0.89 seconds to deallocate network for instance. 2021-09-10 17:01:18.287 4076 3051095191800 MainThread INFO nova.scheduler.client.report [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance 9147ea0e-c0cc-436f-802f-3d7cbccd9529 2021-09-10 17:01:18.287 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [req-3c2c0e77-893b-47ff-9c10-2c627a5b4914 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 11.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:18.287 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529" acquired by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: waited 9.953s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:18.287 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:18.287 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529-events" released by "nova.compute.manager.InstanceEvents.clear_events_for_instance.._clear_events" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:18.287 4076 3051113294136 MainThread ERROR nova.compute.manager [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Setting instance vm_state to ERROR: NameError: name 'instant' is not defined 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Traceback (most recent call last): 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] self._delete_instance(context, instance, bdms) 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] NameError: name 'instant' is not defined 2021-09-10 17:01:18.287 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] 2021-09-10 17:01:18.348 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Setting boot order for instance: instance-00000047: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:01:18.348 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "9147ea0e-c0cc-436f-802f-3d7cbccd9529" released by "nova.compute.manager.ComputeManager.terminate_instance..do_terminate_instance" :: held 0.063s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:18.394 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:01:18.410 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': '8579CA75-E9F7-4911-88C0-CD3F1B3625D4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14373), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:18.441 4076 3051113294136 MainThread INFO nova.compute.manager [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Successfully reverted task state from spawning on failure for instance. 2021-09-10 17:01:18.441 4076 3051113294136 MainThread ERROR oslo_messaging.rpc.server [req-d44fa3a2-70a5-44f3-9958-54094ab2a069 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Exception during message handling: NameError: name 'instant' is not defined 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server Traceback (most recent call last): 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\server.py", line 165, in _process_incoming 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 309, in dispatch 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_messaging\rpc\dispatcher.py", line 229, in _do_dispatch 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server result = func(ctxt, **new_args) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 72, in wrapped 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server context, exc, binary) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\exception_wrapper.py", line 63, in wrapped 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return f(self, context, *args, **kw) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 184, in decorated_function 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 154, in decorated_function 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\utils.py", line 1433, in decorated_function 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 212, in decorated_function 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 200, in decorated_function 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3106, in terminate_instance 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server do_terminate_instance(instance, bdms) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_concurrency\lockutils.py", line 360, in inner 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server return f(*args, **kwargs) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3104, in do_terminate_instance 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self._set_instance_obj_error_state(instance) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self.force_reraise() 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server raise self.value 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3094, in do_terminate_instance 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self._delete_instance(context, instance, bdms) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server File "c:\openstack\build\nova\nova\compute\manager.py", line 3028, in _delete_instance 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server self._power_off_instance(instant, clean_shutdown=True) 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server NameError: name 'instant' is not defined 2021-09-10 17:01:18.441 4076 3051113294136 MainThread TRACE oslo_messaging.rpc.server 2021-09-10 17:01:18.538 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Successfully created port: 918589f1-37ce-452a-a76d-f09f8b62f44e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:18.552 4076 3051095192328 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:18.568 4076 3051095192328 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:18.568 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:18.568 4076 3051095191800 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:18.568 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8579CA75-E9F7-4911-88C0-CD3F1B3625D4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112013), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:18.568 4076 3051113294840 MainThread DEBUG os_win._utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1738283634185791 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:18.568 4076 3051095192328 MainThread DEBUG nova.compute.manager [req-a1869b47-31dc-48d7-a3ec-845acca27925 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:18.568 4076 3051095192328 MainThread DEBUG nova.virt.hyperv.vmops [req-a1869b47-31dc-48d7-a3ec-845acca27925 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:18.629 4076 3051095192328 MainThread DEBUG nova.compute.manager [req-a1869b47-31dc-48d7-a3ec-845acca27925 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:18.660 4076 3051095192328 MainThread INFO nova.compute.manager [req-a1869b47-31dc-48d7-a3ec-845acca27925 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:19.270 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Updated VIF entry in instance network info cache for port 4b0bfd3a-5b69-494a-9940-79c70157a96d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:19.270 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9147ea0e-c0cc-436f-802f-3d7cbccd9529] Updating instance_info_cache with network_info: [{"id": "4b0bfd3a-5b69-494a-9940-79c70157a96d", "address": "fa:16:3e:c3:ce:26", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4b0bfd3a-5b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:19.285 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [req-1aa1613f-41a4-4ccd-8213-9decb68bc3e0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9147ea0e-c0cc-436f-802f-3d7cbccd9529" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:19.599 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': '612BB3F4-DD3E-49B6-AB0F-757ED85CA9F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20226), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:19.725 4076 3051095192680 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:19.725 4076 3051095192680 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:19.725 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:19.725 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:19.725 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '612BB3F4-DD3E-49B6-AB0F-757ED85CA9F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110231), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:19.725 4076 3051113294840 MainThread DEBUG os_win._utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3308491706848145 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:19.751 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-4cb15675-acc2-4526-8082-5815e06d8d30 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:19.751 4076 3051095192680 MainThread DEBUG nova.virt.hyperv.vmops [req-4cb15675-acc2-4526-8082-5815e06d8d30 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:19.800 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-4cb15675-acc2-4526-8082-5815e06d8d30 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:19.832 4076 3051095192680 MainThread INFO nova.compute.manager [req-4cb15675-acc2-4526-8082-5815e06d8d30 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:20.628 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Successfully updated port: 918589f1-37ce-452a-a76d-f09f8b62f44e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:20.644 4076 3051112928888 GreenThread-61 DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Acquired lock "refresh_cache-e55fb7bd-490b-4fc5-9304-262050287269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:20.644 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:20.755 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': '05AFBA0B-32C8-4327-A3CE-3D148EFD6176', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20069), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:20.876 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:20.886 4076 3051095192328 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:20.886 4076 3051095192328 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:20.886 4076 3051095192680 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:20.886 4076 3051095192680 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:20.901 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-eb50bdce-5d06-4fa7-a358-825d9fbe47e8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:20.901 4076 3051095192680 MainThread DEBUG nova.virt.hyperv.vmops [req-eb50bdce-5d06-4fa7-a358-825d9fbe47e8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:20.901 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '05AFBA0B-32C8-4327-A3CE-3D148EFD6176', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110949), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:20.901 4076 3051113294840 MainThread DEBUG os_win._utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.50705623626709 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:20.959 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-eb50bdce-5d06-4fa7-a358-825d9fbe47e8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:20.988 4076 3051095192680 MainThread INFO nova.compute.manager [req-eb50bdce-5d06-4fa7-a358-825d9fbe47e8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:21.925 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': 'E4A0E496-B557-498B-9D98-BC1A6008456B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16246), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:22.054 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:22.054 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:22.054 4076 3051095192680 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:22.054 4076 3051095192680 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:22.068 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E4A0E496-B557-498B-9D98-BC1A6008456B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111731), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:22.068 4076 3051113294840 MainThread DEBUG os_win._utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.673969030380249 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:22.085 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-cdea58db-d2ba-4886-9459-ffefa3674328 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:22.085 4076 3051095192680 MainThread DEBUG nova.virt.hyperv.vmops [req-cdea58db-d2ba-4886-9459-ffefa3674328 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:22.133 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-cdea58db-d2ba-4886-9459-ffefa3674328 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:22.160 4076 3051095192680 MainThread INFO nova.compute.manager [req-cdea58db-d2ba-4886-9459-ffefa3674328 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:22.160 4076 3051112928888 GreenThread-61 DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Updating instance_info_cache with network_info: [{"id": "918589f1-37ce-452a-a76d-f09f8b62f44e", "address": "fa:16:3e:0e:1d:79", "network": {"id": "4ea0beb7-7806-41ea-b250-a4b20b1944e8", "bridge": null, "label": "tempest-ServerMetadataNegativeTestJSON-79328095-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0c42eab1e5884dc19398c04cdcea9edd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap918589f1-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:22.200 4076 3051112928888 GreenThread-61 DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Releasing lock "refresh_cache-e55fb7bd-490b-4fc5-9304-262050287269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:22.200 4076 3051112928888 GreenThread-61 DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Instance network_info: |[{"id": "918589f1-37ce-452a-a76d-f09f8b62f44e", "address": "fa:16:3e:0e:1d:79", "network": {"id": "4ea0beb7-7806-41ea-b250-a4b20b1944e8", "bridge": null, "label": "tempest-ServerMetadataNegativeTestJSON-79328095-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0c42eab1e5884dc19398c04cdcea9edd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap918589f1-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:22.200 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Preparing to wait for external event network-vif-plugged-918589f1-37ce-452a-a76d-f09f8b62f44e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:22.200 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:22.200 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:22.200 4076 3051095192152 MainThread DEBUG os_win.utils.compute.vmutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Creating VM instance-0000004a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:22.837 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:22.859 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:22.974 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:22.974 4076 3051113294136 MainThread DEBUG nova.virt.hardware [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:22.974 4076 3051113294136 MainThread INFO nova.compute.claims [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Claim successful on node n-h1-808474-1 2021-09-10 17:01:23.082 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': 'A1A575EF-EC30-462F-9EF4-610BA58E693E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15140), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:23.113 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:23.144 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:23.159 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:23.227 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:23.227 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:23.227 4076 3051112928888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:23.227 4076 3051112928888 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:23.246 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A1A575EF-EC30-462F-9EF4-610BA58E693E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114012), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:23.246 4076 3051113294840 MainThread DEBUG os_win._utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.851964473724365 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:23.246 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Received event network-changed-918589f1-37ce-452a-a76d-f09f8b62f44e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:23.246 4076 3051095192680 MainThread DEBUG nova.compute.manager [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Refreshing instance network info cache due to event network-changed-918589f1-37ce-452a-a76d-f09f8b62f44e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:23.246 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-e55fb7bd-490b-4fc5-9304-262050287269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:23.246 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Refreshing network info cache for port 918589f1-37ce-452a-a76d-f09f8b62f44e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:23.246 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-207a6d99-2a04-4694-88e2-f08db2f799d8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:23.246 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-207a6d99-2a04-4694-88e2-f08db2f799d8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:23.316 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-207a6d99-2a04-4694-88e2-f08db2f799d8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:23.348 4076 3051112928888 MainThread INFO nova.compute.manager [req-207a6d99-2a04-4694-88e2-f08db2f799d8 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:23.395 4076 3051113294136 MainThread DEBUG nova.compute.provider_tree [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:23.414 4076 3051113294136 MainThread DEBUG nova.scheduler.client.report [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:23.414 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:23.414 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:23.543 4076 3051113294136 MainThread DEBUG nova.compute.utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:23.543 4076 3051095192504 GreenThread-62 DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:23.543 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:23.552 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:23.686 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:23.686 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Spawning new instance 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG nova.block_device [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG os_win.utils.pathutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-0000004b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:23.707 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000004b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:23.733 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Resizing VHD C:\openstack\instances\instance-0000004b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:23.926 4076 3051095192504 GreenThread-62 DEBUG nova.policy [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:24.285 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': '0EF5D88D-BB12-4BC0-9298-2CD6408E28CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14693), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:24.403 4076 3051095193208 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:24.403 4076 3051095193208 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:24.403 4076 3051095191800 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:24.418 4076 3051095191800 MainThread INFO nova.compute.manager [-] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] VM Stopped (Lifecycle Event) 2021-09-10 17:01:24.427 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000047\' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000047\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)instance-00000047CBC8E8B7-E407-477B-8433-38520E138EB0%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0EF5D88D-BB12-4BC0-9298-2CD6408E28CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110133), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\nNot enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)\n\n'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0)", 'ErrorSummaryDescription': "'instance-00000047' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000047.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:24.427 4076 3051113294840 MainThread ERROR nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Failed to change vm state of instance-00000047 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. 2021-09-10 17:01:24.427 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Got request to destroy instance 2021-09-10 17:01:24.473 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-041a3436-4797-4e9f-80e1-29ee1774c755 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:24.473 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.vmops [req-041a3436-4797-4e9f-80e1-29ee1774c755 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:24.521 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-041a3436-4797-4e9f-80e1-29ee1774c755 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:24.560 4076 3051095191800 MainThread INFO nova.compute.manager [req-041a3436-4797-4e9f-80e1-29ee1774c755 - - - - -] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:25.305 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Updated VIF entry in instance network info cache for port 918589f1-37ce-452a-a76d-f09f8b62f44e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:25.322 4076 3051095192680 MainThread DEBUG nova.network.neutron [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Updating instance_info_cache with network_info: [{"id": "918589f1-37ce-452a-a76d-f09f8b62f44e", "address": "fa:16:3e:0e:1d:79", "network": {"id": "4ea0beb7-7806-41ea-b250-a4b20b1944e8", "bridge": null, "label": "tempest-ServerMetadataNegativeTestJSON-79328095-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0c42eab1e5884dc19398c04cdcea9edd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap918589f1-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:25.337 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [req-a88c0d0e-cd31-4f13-97e0-53712e846c51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-e55fb7bd-490b-4fc5-9304-262050287269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:26.253 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:01:26.253 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:26.253 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:26.253 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Successfully changed vm state of instance-00000047 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:01:26.253 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Successfully changed state of VM instance-00000047 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:01:26.311 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Waiting for WMI job: {'InstanceID': 'D1AB4AAB-6958-489C-A944-5536DB741C1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15233), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:26.426 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] WMI job succeeded: {'InstanceID': 'D1AB4AAB-6958-489C-A944-5536DB741C1E', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28824), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:01:26.426 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:26.426 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Removing directory: C:\openstack\instances\instance-00000047 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:01:26.446 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:26.446 4076 3051113294840 MainThread ERROR nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Traceback (most recent call last): 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(*args, **kwargs) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] admin_password, network_info, block_device_info) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(self, *args, **kwds) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] should_plug_vifs=False) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'req_state': req_state}) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] time.sleep(sleep_time) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return f(*args, **kwargs) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return self._wait_for_job(job_path) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] error_desc=err_desc) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During handling of the above exception, another exception occurred: 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Traceback (most recent call last): 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] yield resources 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] accel_info=accel_info) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(*args, **kwargs) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] admin_password, network_info, block_device_info) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(self, *args, **kwds) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] should_plug_vifs=False) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'req_state': req_state}) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] time.sleep(sleep_time) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return f(*args, **kwargs) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return self._wait_for_job(job_path) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] error_desc=err_desc) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.446 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.446 4076 3051113294840 MainThread INFO nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Terminating instance 2021-09-10 17:01:26.446 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:26.446 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Got request to destroy instance 2021-09-10 17:01:26.458 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:26.458 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:26.458 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "instance-00000047" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:26.458 4076 3051113294840 MainThread INFO nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:26.458 4076 3051113294840 MainThread DEBUG nova.compute.claims [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:26.458 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:26.504 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Successfully created port: 0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:26.566 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:26.566 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:26.773 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:26.791 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:26.791 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:26.791 4076 3051113294840 MainThread ERROR nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Traceback (most recent call last): 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(*args, **kwargs) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] admin_password, network_info, block_device_info) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(self, *args, **kwds) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] should_plug_vifs=False) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'req_state': req_state}) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] time.sleep(sleep_time) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return f(*args, **kwargs) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return self._wait_for_job(job_path) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] error_desc=err_desc) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] During handling of the above exception, another exception occurred: 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Traceback (most recent call last): 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] accel_info=accel_info) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(*args, **kwargs) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] admin_password, network_info, block_device_info) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return function(self, *args, **kwds) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] should_plug_vifs=False) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'req_state': req_state}) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] time.sleep(sleep_time) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self.force_reraise() 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] raise self.value 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return f(*args, **kwargs) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] return self._wait_for_job(job_path) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] error_desc=err_desc) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. 2021-09-10 17:01:26.791 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 219ee2c7-854a-4265-903f-5d10381fa708] 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Build of instance 219ee2c7-854a-4265-903f-5d10381fa708 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000047' failed to start. Not enough memory in the system to start the virtual machine instance-00000047. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000047' failed to start. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Not enough memory in the system to start the virtual machine instance-00000047 with ram size 1024 megabytes. (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) 'instance-00000047' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CBC8E8B7-E407-477B-8433-38520E138EB0) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:26.808 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:01:29.099 4076 3051095193032 MainThread DEBUG nova.compute.manager [req-61848719-8ee7-4bac-a57d-b66f4ad529f3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Received event network-vif-plugged-918589f1-37ce-452a-a76d-f09f8b62f44e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:29.099 4076 3051095193032 MainThread DEBUG oslo_concurrency.lockutils [req-61848719-8ee7-4bac-a57d-b66f4ad529f3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:29.099 4076 3051095193032 MainThread DEBUG oslo_concurrency.lockutils [req-61848719-8ee7-4bac-a57d-b66f4ad529f3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:29.099 4076 3051095193032 MainThread DEBUG nova.compute.manager [req-61848719-8ee7-4bac-a57d-b66f4ad529f3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Processing event network-vif-plugged-918589f1-37ce-452a-a76d-f09f8b62f44e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:01:29.135 4076 3051095192152 MainThread INFO nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Using config drive for instance 2021-09-10 17:01:29.175 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Successfully updated port: 0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:29.207 4076 3051095192504 GreenThread-62 DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-3111054c-9a20-4851-a793-42fbe94a3375" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:29.207 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:29.207 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:29.239 4076 3051113294840 MainThread INFO nova.compute.manager [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] [instance: 219ee2c7-854a-4265-903f-5d10381fa708] Took 2.44 seconds to deallocate network for instance. 2021-09-10 17:01:29.270 4076 3051095192152 MainThread INFO nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Creating config drive at C:\openstack\instances\instance-0000004a\configdrive.iso 2021-09-10 17:01:29.317 4076 3051095192152 MainThread DEBUG oslo_concurrency.processutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp188was9y execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:01:29.445 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:29.581 4076 3051113294840 MainThread INFO nova.scheduler.client.report [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Deleted allocations for instance 219ee2c7-854a-4265-903f-5d10381fa708 2021-09-10 17:01:29.614 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-3a4689ab-d2e5-48ba-bb3b-2b15d86e5139 3f85143bbedf44eca37aba098b6ccb7d 5f2d0543bdd24962be5e0dceb47ee53e - default default] Lock "219ee2c7-854a-4265-903f-5d10381fa708" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:29.614 4076 3051095192152 MainThread DEBUG oslo_concurrency.processutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp188was9y" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:01:29.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:30.020 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Setting boot order for instance: instance-0000004a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:01:30.070 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:01:30.104 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '510D89C5-734F-4BCA-BE99-49828F59C8F9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14588), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:30.223 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:30.223 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:30.239 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:30.239 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:30.239 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '510D89C5-734F-4BCA-BE99-49828F59C8F9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112424), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:30.239 4076 3051095192152 MainThread DEBUG os_win._utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16834354400634766 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:30.254 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-1b304493-f61e-4d12-8024-557709f43642 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:30.254 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-1b304493-f61e-4d12-8024-557709f43642 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:30.290 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-1b304493-f61e-4d12-8024-557709f43642 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:30.347 4076 3051113294840 MainThread INFO nova.compute.manager [req-1b304493-f61e-4d12-8024-557709f43642 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:30.644 4076 3051095192504 GreenThread-62 DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Updating instance_info_cache with network_info: [{"id": "0fea0e67-b5a1-496e-90ef-6d8545f2b5f8", "address": "fa:16:3e:8a:a7:cf", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0fea0e67-b5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:30.661 4076 3051095192504 GreenThread-62 DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-3111054c-9a20-4851-a793-42fbe94a3375" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:30.661 4076 3051095192504 GreenThread-62 DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Instance network_info: |[{"id": "0fea0e67-b5a1-496e-90ef-6d8545f2b5f8", "address": "fa:16:3e:8a:a7:cf", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0fea0e67-b5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:30.661 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Preparing to wait for external event network-vif-plugged-0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:30.661 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:30.661 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:30.661 4076 3051113294136 MainThread DEBUG os_win.utils.compute.vmutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-0000004b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:31.265 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '775C3F5D-2E65-44E8-A175-C3E7BBFE4EE6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15154), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:31.413 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '775C3F5D-2E65-44E8-A175-C3E7BBFE4EE6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124380), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:31.413 4076 3051095192152 MainThread DEBUG os_win._utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3428213596343994 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:31.413 4076 3051112928888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:31.413 4076 3051112928888 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:31.413 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:31.413 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:31.442 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-c5421720-a772-48b7-9399-96119537ce82 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:31.442 4076 3051112928888 MainThread DEBUG nova.virt.hyperv.vmops [req-c5421720-a772-48b7-9399-96119537ce82 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:31.482 4076 3051112928888 MainThread DEBUG nova.compute.manager [req-c5421720-a772-48b7-9399-96119537ce82 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:31.519 4076 3051112928888 MainThread INFO nova.compute.manager [req-c5421720-a772-48b7-9399-96119537ce82 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:31.586 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:32.441 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '3F03E946-A901-4459-ABDD-6134143F6419', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14493), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:32.489 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Received event network-changed-0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:32.489 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Refreshing instance network info cache due to event network-changed-0fea0e67-b5a1-496e-90ef-6d8545f2b5f8. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:32.489 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-3111054c-9a20-4851-a793-42fbe94a3375" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:32.489 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Refreshing network info cache for port 0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:32.572 4076 3051095193208 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:32.572 4076 3051095193208 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:32.572 4076 3051095191800 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:32.572 4076 3051095191800 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:32.598 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3F03E946-A901-4459-ABDD-6134143F6419', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111499), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:32.598 4076 3051095192152 MainThread DEBUG os_win._utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.527164936065674 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:32.613 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-c0248a90-846f-4a6f-9e45-4fb499aadb21 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:32.613 4076 3051095191800 MainThread DEBUG nova.virt.hyperv.vmops [req-c0248a90-846f-4a6f-9e45-4fb499aadb21 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:32.660 4076 3051095191800 MainThread DEBUG nova.compute.manager [req-c0248a90-846f-4a6f-9e45-4fb499aadb21 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:32.676 4076 3051095191800 MainThread INFO nova.compute.manager [req-c0248a90-846f-4a6f-9e45-4fb499aadb21 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:32.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:32.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:01:32.994 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:32.994 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:01:33.643 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '88B9D7B5-0BF8-403D-AE07-47962D639479', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15935), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:33.771 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:33.771 4076 3051095192680 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:33.771 4076 3051095193208 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:33.771 4076 3051095193208 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:33.800 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '88B9D7B5-0BF8-403D-AE07-47962D639479', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110591), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:33.800 4076 3051095192152 MainThread DEBUG os_win._utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.72963285446167 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:33.816 4076 3051095193208 MainThread DEBUG nova.compute.manager [req-9736ebae-00bc-4bc7-aa29-117dc4dd8895 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:33.816 4076 3051095193208 MainThread DEBUG nova.virt.hyperv.vmops [req-9736ebae-00bc-4bc7-aa29-117dc4dd8895 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:33.833 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Updated VIF entry in instance network info cache for port 0fea0e67-b5a1-496e-90ef-6d8545f2b5f8. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:33.833 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Updating instance_info_cache with network_info: [{"id": "0fea0e67-b5a1-496e-90ef-6d8545f2b5f8", "address": "fa:16:3e:8a:a7:cf", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0fea0e67-b5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:33.833 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-f7580f07-7a8e-4b9c-a84a-a894fb365da6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-3111054c-9a20-4851-a793-42fbe94a3375" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:33.867 4076 3051095193208 MainThread DEBUG nova.compute.manager [req-9736ebae-00bc-4bc7-aa29-117dc4dd8895 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:33.894 4076 3051095193208 MainThread INFO nova.compute.manager [req-9736ebae-00bc-4bc7-aa29-117dc4dd8895 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:34.181 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:34.181 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:34.202 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:01:34.202 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:34.202 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:01:34.202 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:34.825 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '965FAC62-A1F5-4459-BFD9-7E3E9F346FE7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14892), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:34.957 4076 3051113294840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:34.957 4076 3051113294840 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:34.957 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:34.957 4076 3051112928888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:34.976 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '965FAC62-A1F5-4459-BFD9-7E3E9F346FE7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111063), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:34.976 4076 3051095192152 MainThread DEBUG os_win._utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.906073331832886 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:35.005 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-21da335f-0f14-4693-945c-f21ab6d3f146 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:35.005 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-21da335f-0f14-4693-945c-f21ab6d3f146 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:35.038 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:35.038 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:35.054 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-21da335f-0f14-4693-945c-f21ab6d3f146 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:35.085 4076 3051113294840 MainThread INFO nova.compute.manager [req-21da335f-0f14-4693-945c-f21ab6d3f146 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:35.474 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:35.504 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:35.738 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:35.738 4076 3051095191624 MainThread DEBUG nova.virt.hardware [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:35.738 4076 3051095191624 MainThread INFO nova.compute.claims [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Claim successful on node n-h1-808474-1 2021-09-10 17:01:35.831 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:35.879 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:01:35.915 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:01:35.915 4076 3051095191624 MainThread DEBUG nova.compute.provider_tree [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:01:35.941 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:01:35.974 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:01:35.974 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.hostops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:35.989 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.hostops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:36.007 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '511B9F1A-1956-49A0-A2FB-1ABA8DAAC476', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18874), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:36.116 4076 3051095191976 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:36.116 4076 3051095191976 MainThread INFO nova.compute.manager [-] [instance: e55fb7bd-490b-4fc5-9304-262050287269] VM Stopped (Lifecycle Event) 2021-09-10 17:01:36.116 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:36.144 4076 3051095192504 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:36.160 4076 3051095191976 MainThread DEBUG nova.compute.manager [req-bf20dbba-3743-4eb1-9e5a-a827be9d7af3 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:36.160 4076 3051095191976 MainThread DEBUG nova.virt.hyperv.vmops [req-bf20dbba-3743-4eb1-9e5a-a827be9d7af3 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:36.160 4076 3051095192152 MainThread ERROR os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004a\' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-39487812476410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)instance-0000004aD0645C31-6BD8-4E2B-ABD6-394878124764%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '511B9F1A-1956-49A0-A2FB-1ABA8DAAC476', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\nNot enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)\n\n'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764)", 'ErrorSummaryDescription': "'instance-0000004a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:36.160 4076 3051095192152 MainThread ERROR nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Failed to change vm state of instance-0000004a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. 2021-09-10 17:01:36.160 4076 3051095192152 MainThread INFO nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Got request to destroy instance 2021-09-10 17:01:36.212 4076 3051095191624 MainThread DEBUG nova.compute.provider_tree [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:36.212 4076 3051095191976 MainThread DEBUG nova.compute.manager [req-bf20dbba-3743-4eb1-9e5a-a827be9d7af3 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:36.227 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:36.227 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:36.227 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:36.238 4076 3051095191976 MainThread INFO nova.compute.manager [req-bf20dbba-3743-4eb1-9e5a-a827be9d7af3 - - - - -] [instance: e55fb7bd-490b-4fc5-9304-262050287269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:36.307 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:36.316 4076 3051095191976 GreenThread-63 DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:36.316 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:36.478 4076 3051095191976 GreenThread-63 DEBUG nova.policy [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0574bfa519834ae2b8ce81f3a560b957', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58b055dc3ac4474788971e8148bec63e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:36.478 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:36.498 4076 3051095191624 MainThread INFO nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Spawning new instance 2021-09-10 17:01:36.498 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:36.498 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:36.498 4076 3051095191624 MainThread DEBUG nova.block_device [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:36.498 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:36.498 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:36.519 4076 3051095191624 MainThread DEBUG os_win.utils.pathutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Creating directory: C:\openstack\instances\instance-00000048 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:36.519 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000048\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:36.534 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Resizing VHD C:\openstack\instances\instance-00000048\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:36.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:36.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:37.082 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Successfully created port: e7393ff8-ad34-4c92-82fb-ace93bbd1e91 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:37.794 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:37.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:37.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:37.800 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:01:37.800 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:37.832 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:37.847 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:37.863 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:37.896 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Successfully updated port: e7393ff8-ad34-4c92-82fb-ace93bbd1e91 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:37.925 4076 3051095191976 GreenThread-63 DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Acquired lock "refresh_cache-9015b51e-d705-4694-8c71-5322dc51aa39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:37.925 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:38.019 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.036 4076 3051095192856 MainThread DEBUG nova.virt.hardware [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:38.036 4076 3051095192856 MainThread INFO nova.compute.claims [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Claim successful on node n-h1-808474-1 2021-09-10 17:01:38.036 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:38.085 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:01:38.085 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.085 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.085 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2175MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:01:38.114 4076 3051095192152 MainThread DEBUG os_win.utils.compute.vmutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Successfully changed vm state of instance-0000004a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:01:38.114 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Successfully changed state of VM instance-0000004a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:01:38.163 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.hostops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:38.195 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Waiting for WMI job: {'InstanceID': '7CDC2C97-9991-44CB-B9D6-3DED0C4EB475', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15840), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:38.195 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.hostops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:38.227 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-5016a3d6-7aa1-4b40-809f-6400dbaf1265 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Received event network-vif-plugged-0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:38.227 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [req-5016a3d6-7aa1-4b40-809f-6400dbaf1265 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.238 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [req-5016a3d6-7aa1-4b40-809f-6400dbaf1265 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.238 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-5016a3d6-7aa1-4b40-809f-6400dbaf1265 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Processing event network-vif-plugged-0fea0e67-b5a1-496e-90ef-6d8545f2b5f8 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:01:38.270 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Received event network-changed-e7393ff8-ad34-4c92-82fb-ace93bbd1e91 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:38.270 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Refreshing instance network info cache due to event network-changed-e7393ff8-ad34-4c92-82fb-ace93bbd1e91. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:38.270 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Using config drive for instance 2021-09-10 17:01:38.307 4076 3051095192152 MainThread DEBUG os_win.utils.jobutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] WMI job succeeded: {'InstanceID': '7CDC2C97-9991-44CB-B9D6-3DED0C4EB475', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31864), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:01:38.307 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.307 4076 3051095192152 MainThread DEBUG os_win.utils.pathutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Removing directory: C:\openstack\instances\instance-0000004a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:01:38.307 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.307 4076 3051095192152 MainThread ERROR nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Traceback (most recent call last): 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(*args, **kwargs) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] admin_password, network_info, block_device_info) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(self, *args, **kwds) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] should_plug_vifs=False) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'req_state': req_state}) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] time.sleep(sleep_time) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return f(*args, **kwargs) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return self._wait_for_job(job_path) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] error_desc=err_desc) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] During handling of the above exception, another exception occurred: 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Traceback (most recent call last): 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] yield resources 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] accel_info=accel_info) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(*args, **kwargs) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] admin_password, network_info, block_device_info) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(self, *args, **kwds) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] should_plug_vifs=False) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'req_state': req_state}) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] time.sleep(sleep_time) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return f(*args, **kwargs) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return self._wait_for_job(job_path) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] error_desc=err_desc) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:38.307 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:38.332 4076 3051095192152 MainThread INFO nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Terminating instance 2021-09-10 17:01:38.332 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:38.332 4076 3051095192152 MainThread INFO nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Got request to destroy instance 2021-09-10 17:01:38.353 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:38.353 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.353 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "instance-0000004a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.353 4076 3051095192152 MainThread INFO nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:38.353 4076 3051095192152 MainThread DEBUG nova.compute.claims [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:38.425 4076 3051095192856 MainThread DEBUG nova.compute.provider_tree [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:38.458 4076 3051095192856 MainThread DEBUG nova.scheduler.client.report [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:38.458 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.458 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:38.473 4076 3051095191976 GreenThread-63 DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Updating instance_info_cache with network_info: [{"id": "e7393ff8-ad34-4c92-82fb-ace93bbd1e91", "address": "fa:16:3e:61:ea:a9", "network": {"id": "5f474ded-ff91-42f7-a10b-055b13eaa82a", "bridge": null, "label": "tempest-ServersNegativeTestJSON-2099773424-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "58b055dc3ac4474788971e8148bec63e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape7393ff8-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:38.473 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.488 4076 3051095191976 GreenThread-63 DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Releasing lock "refresh_cache-9015b51e-d705-4694-8c71-5322dc51aa39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:38.488 4076 3051095191976 GreenThread-63 DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Instance network_info: |[{"id": "e7393ff8-ad34-4c92-82fb-ace93bbd1e91", "address": "fa:16:3e:61:ea:a9", "network": {"id": "5f474ded-ff91-42f7-a10b-055b13eaa82a", "bridge": null, "label": "tempest-ServersNegativeTestJSON-2099773424-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "58b055dc3ac4474788971e8148bec63e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape7393ff8-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:38.488 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Preparing to wait for external event network-vif-plugged-e7393ff8-ad34-4c92-82fb-ace93bbd1e91 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:38.488 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.488 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.488 4076 3051095191624 MainThread DEBUG os_win.utils.compute.vmutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Creating VM instance-00000048 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:38.488 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9015b51e-d705-4694-8c71-5322dc51aa39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:38.488 4076 3051095193384 MainThread DEBUG nova.network.neutron [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Refreshing network info cache for port e7393ff8-ad34-4c92-82fb-ace93bbd1e91 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:38.554 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Creating config drive at C:\openstack\instances\instance-0000004b\configdrive.iso 2021-09-10 17:01:38.587 4076 3051113294136 MainThread DEBUG oslo_concurrency.processutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpj7ujgp_f execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:01:38.760 4076 3051095192856 MainThread DEBUG nova.compute.utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:38.760 4076 3051095191976 GreenThread-64 DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:38.760 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:38.784 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e55fb7bd-490b-4fc5-9304-262050287269 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 3111054c-9a20-4851-a793-42fbe94a3375 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9015b51e-d705-4694-8c71-5322dc51aa39 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance b0e61418-3187-4180-bea6-4e63a7494fbc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 16 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:01:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=16896MB phys_disk=79GB used_disk=16GB total_vcpus=6 used_vcpus=16 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:01:38.881 4076 3051113294136 MainThread DEBUG oslo_concurrency.processutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpj7ujgp_f" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:01:38.898 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:38.935 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:38.957 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:38.957 4076 3051095192856 MainThread INFO nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Spawning new instance 2021-09-10 17:01:38.981 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.981 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.981 4076 3051095192856 MainThread DEBUG nova.block_device [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:38.981 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:38.981 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:38.993 4076 3051095192856 MainThread DEBUG os_win.utils.pathutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating directory: C:\openstack\instances\instance-0000004c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:38.993 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000004c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:39.004 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Resizing VHD C:\openstack\instances\instance-0000004c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:39.051 4076 3051095191976 GreenThread-64 DEBUG nova.policy [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b207fe385eb647a4acf0331024dcf245', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '00a75e521b5f4d89ab74f2701a36da12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:39.178 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:39.192 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:39.192 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:01:39.192 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.719s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:39.192 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.844s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:39.302 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.hostops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:39.318 4076 3051095193384 MainThread DEBUG nova.network.neutron [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Updated VIF entry in instance network info cache for port e7393ff8-ad34-4c92-82fb-ace93bbd1e91. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:39.318 4076 3051095193384 MainThread DEBUG nova.network.neutron [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Updating instance_info_cache with network_info: [{"id": "e7393ff8-ad34-4c92-82fb-ace93bbd1e91", "address": "fa:16:3e:61:ea:a9", "network": {"id": "5f474ded-ff91-42f7-a10b-055b13eaa82a", "bridge": null, "label": "tempest-ServersNegativeTestJSON-2099773424-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "58b055dc3ac4474788971e8148bec63e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape7393ff8-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:39.318 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.hostops [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:39.318 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-0000004b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:01:39.338 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [req-757051fb-6245-4f39-b061-29dfa7972d69 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9015b51e-d705-4694-8c71-5322dc51aa39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:39.383 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:01:39.417 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'A4177898-3793-4AAD-8A5D-FF86259A86B9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17961), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:39.552 4076 3051095192152 MainThread DEBUG nova.compute.provider_tree [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:39.552 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.scheduler.client.report [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:39.567 4076 3051095192152 MainThread ERROR nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Traceback (most recent call last): 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(*args, **kwargs) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] admin_password, network_info, block_device_info) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(self, *args, **kwds) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] should_plug_vifs=False) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'req_state': req_state}) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] time.sleep(sleep_time) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return f(*args, **kwargs) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return self._wait_for_job(job_path) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] error_desc=err_desc) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] During handling of the above exception, another exception occurred: 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Traceback (most recent call last): 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] accel_info=accel_info) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(*args, **kwargs) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] admin_password, network_info, block_device_info) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return function(self, *args, **kwds) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] should_plug_vifs=False) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'req_state': req_state}) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] time.sleep(sleep_time) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self.force_reraise() 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] raise self.value 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return f(*args, **kwargs) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] return self._wait_for_job(job_path) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] error_desc=err_desc) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. 2021-09-10 17:01:39.567 4076 3051095192152 MainThread TRACE nova.compute.manager [instance: e55fb7bd-490b-4fc5-9304-262050287269] 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.compute.utils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Build of instance e55fb7bd-490b-4fc5-9304-262050287269 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000004a' failed to start. Not enough memory in the system to start the virtual machine instance-0000004a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004a' failed to start. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Not enough memory in the system to start the virtual machine instance-0000004a with ram size 1024 megabytes. (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) 'instance-0000004a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D0645C31-6BD8-4E2B-ABD6-394878124764) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:39.567 4076 3051095192152 MainThread DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:01:39.567 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A4177898-3793-4AAD-8A5D-FF86259A86B9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122319), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:39.567 4076 3051113294136 MainThread DEBUG os_win._utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18402600288391113 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:39.567 4076 3051095194440 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:39.567 4076 3051095194440 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:39.591 4076 3051095193384 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:39.591 4076 3051095193384 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:39.611 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-7eb41ea9-c0d2-49aa-950d-aec6544f1f91 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:39.614 4076 3051095193384 MainThread DEBUG nova.virt.hyperv.vmops [req-7eb41ea9-c0d2-49aa-950d-aec6544f1f91 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:39.660 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-7eb41ea9-c0d2-49aa-950d-aec6544f1f91 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:39.678 4076 3051095193384 MainThread INFO nova.compute.manager [req-7eb41ea9-c0d2-49aa-950d-aec6544f1f91 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:40.131 4076 3051095192152 MainThread DEBUG nova.network.neutron [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:40.153 4076 3051095192152 MainThread INFO nova.compute.manager [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] [instance: e55fb7bd-490b-4fc5-9304-262050287269] Took 0.59 seconds to deallocate network for instance. 2021-09-10 17:01:40.345 4076 3051095192152 MainThread INFO nova.scheduler.client.report [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Deleted allocations for instance e55fb7bd-490b-4fc5-9304-262050287269 2021-09-10 17:01:40.394 4076 3051095192152 MainThread DEBUG oslo_concurrency.lockutils [req-e6b6c0f4-59ea-453f-ac9c-a5db67ece1b0 c5eaa901834b4ee885c2b6a9ce410303 0c42eab1e5884dc19398c04cdcea9edd - default default] Lock "e55fb7bd-490b-4fc5-9304-262050287269" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.828s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:40.597 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '40D703B9-5662-4D6C-BC94-DC014870BA70', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14579), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:40.726 4076 3051095192152 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:40.726 4076 3051095192152 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:40.726 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:40.726 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:40.738 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '40D703B9-5662-4D6C-BC94-DC014870BA70', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110399), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:40.738 4076 3051113294136 MainThread DEBUG os_win._utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3553380966186523 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:40.738 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-4b920c7d-4653-43d6-bccd-831b7d956f6a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:40.738 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-4b920c7d-4653-43d6-bccd-831b7d956f6a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:40.800 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-4b920c7d-4653-43d6-bccd-831b7d956f6a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:40.815 4076 3051095192152 MainThread INFO nova.compute.manager [req-4b920c7d-4653-43d6-bccd-831b7d956f6a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:40.963 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Successfully created port: 7abbc9b3-0ae7-4623-a681-03cc6e640f08 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:41.175 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:41.191 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:41.191 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:01:41.771 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'F161A832-4DDA-4153-8634-CC99874A52EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14966), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:41.800 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:01:41.800 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:01:41.819 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:01:41.913 4076 3051095192152 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:41.913 4076 3051095192152 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:41.913 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:41.913 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:41.913 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F161A832-4DDA-4153-8634-CC99874A52EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111669), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:41.913 4076 3051113294136 MainThread DEBUG os_win._utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.530336618423462 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:41.913 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-31fec8d6-0d16-49a9-a8fc-d0121b76febe - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:41.913 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-31fec8d6-0d16-49a9-a8fc-d0121b76febe - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:41.980 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-31fec8d6-0d16-49a9-a8fc-d0121b76febe - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:42.056 4076 3051095192152 MainThread INFO nova.compute.manager [req-31fec8d6-0d16-49a9-a8fc-d0121b76febe - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:42.927 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Successfully updated port: 7abbc9b3-0ae7-4623-a681-03cc6e640f08 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:42.945 4076 3051095191976 GreenThread-64 DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Acquired lock "refresh_cache-b0e61418-3187-4180-bea6-4e63a7494fbc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:42.945 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:42.959 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '4EAA3D97-A5FE-4329-9C64-E7208F495FD4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16774), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:43.085 4076 3051095194440 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:43.085 4076 3051095194440 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:43.085 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:43.085 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:43.098 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4EAA3D97-A5FE-4329-9C64-E7208F495FD4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111923), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:43.098 4076 3051113294136 MainThread DEBUG os_win._utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7156033515930176 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:43.124 4076 3051095194440 MainThread DEBUG nova.compute.manager [req-04f2ae6e-d906-4a9e-9b16-f2c9bc559470 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:43.124 4076 3051095194440 MainThread DEBUG nova.virt.hyperv.vmops [req-04f2ae6e-d906-4a9e-9b16-f2c9bc559470 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:43.124 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:43.161 4076 3051095194440 MainThread DEBUG nova.compute.manager [req-04f2ae6e-d906-4a9e-9b16-f2c9bc559470 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:43.194 4076 3051095194440 MainThread INFO nova.compute.manager [req-04f2ae6e-d906-4a9e-9b16-f2c9bc559470 - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:43.926 4076 3051095194440 MainThread DEBUG nova.compute.manager [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Received event network-changed-7abbc9b3-0ae7-4623-a681-03cc6e640f08 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:43.926 4076 3051095194440 MainThread DEBUG nova.compute.manager [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Refreshing instance network info cache due to event network-changed-7abbc9b3-0ae7-4623-a681-03cc6e640f08. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:44.134 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '232AEF38-8AB3-48A9-BE1E-DDD4AC0D84CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15775), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:44.267 4076 3051095192152 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:44.267 4076 3051095192152 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:44.267 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:44.267 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:44.285 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '232AEF38-8AB3-48A9-BE1E-DDD4AC0D84CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114004), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:44.285 4076 3051113294136 MainThread DEBUG os_win._utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.90266489982605 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:44.301 4076 3051095191976 GreenThread-64 DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Updating instance_info_cache with network_info: [{"id": "7abbc9b3-0ae7-4623-a681-03cc6e640f08", "address": "fa:16:3e:fa:3e:0d", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7abbc9b3-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:44.301 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-656e4159-6704-4768-a593-d1f8e803c10a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:44.301 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-656e4159-6704-4768-a593-d1f8e803c10a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:44.322 4076 3051095191976 GreenThread-64 DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Releasing lock "refresh_cache-b0e61418-3187-4180-bea6-4e63a7494fbc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:44.322 4076 3051095191976 GreenThread-64 DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Instance network_info: |[{"id": "7abbc9b3-0ae7-4623-a681-03cc6e640f08", "address": "fa:16:3e:fa:3e:0d", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7abbc9b3-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:44.322 4076 3051095194440 MainThread DEBUG oslo_concurrency.lockutils [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-b0e61418-3187-4180-bea6-4e63a7494fbc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:44.322 4076 3051095194440 MainThread DEBUG nova.network.neutron [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Refreshing network info cache for port 7abbc9b3-0ae7-4623-a681-03cc6e640f08 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:44.322 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Preparing to wait for external event network-vif-plugged-7abbc9b3-0ae7-4623-a681-03cc6e640f08 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:44.322 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:44.322 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:44.335 4076 3051095192856 MainThread DEBUG os_win.utils.compute.vmutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating VM instance-0000004c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:44.348 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-656e4159-6704-4768-a593-d1f8e803c10a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:44.378 4076 3051095192152 MainThread INFO nova.compute.manager [req-656e4159-6704-4768-a593-d1f8e803c10a - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:44.604 4076 3051095191976 MainThread DEBUG nova.compute.manager [req-96599580-da3e-4671-a65b-7a0e1be8c05d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Received event network-vif-plugged-e7393ff8-ad34-4c92-82fb-ace93bbd1e91 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:44.604 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [req-96599580-da3e-4671-a65b-7a0e1be8c05d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:44.604 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [req-96599580-da3e-4671-a65b-7a0e1be8c05d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:44.604 4076 3051095191976 MainThread DEBUG nova.compute.manager [req-96599580-da3e-4671-a65b-7a0e1be8c05d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Processing event network-vif-plugged-e7393ff8-ad34-4c92-82fb-ace93bbd1e91 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:01:44.653 4076 3051095191624 MainThread INFO nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Using config drive for instance 2021-09-10 17:01:44.813 4076 3051095191624 MainThread INFO nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Creating config drive at C:\openstack\instances\instance-00000048\configdrive.iso 2021-09-10 17:01:44.865 4076 3051095191624 MainThread DEBUG oslo_concurrency.processutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000048\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8xw7x1sh execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:01:45.167 4076 3051095191624 MainThread DEBUG oslo_concurrency.processutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000048\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8xw7x1sh" returned: 0 in 0.296s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:01:45.333 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '13C36B7F-C8AF-4C47-A2FB-B7B3357D347E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15042), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:45.372 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:45.488 4076 3051095194792 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:45.488 4076 3051095194792 MainThread INFO nova.compute.manager [-] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] VM Stopped (Lifecycle Event) 2021-09-10 17:01:45.488 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:45.488 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:45.488 4076 3051113294136 MainThread ERROR os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004b\' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)instance-0000004b41166C1C-9973-467F-A6C9-EF4EE16154B7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '13C36B7F-C8AF-4C47-A2FB-B7B3357D347E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122132), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\nNot enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)\n\n'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7)", 'ErrorSummaryDescription': "'instance-0000004b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:45.488 4076 3051113294136 MainThread ERROR nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-0000004b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. 2021-09-10 17:01:45.488 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Got request to destroy instance 2021-09-10 17:01:45.488 4076 3051095194792 MainThread DEBUG nova.compute.manager [req-fdd4476a-1c93-4bae-9120-260285d6769d - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:45.488 4076 3051095194792 MainThread DEBUG nova.virt.hyperv.vmops [req-fdd4476a-1c93-4bae-9120-260285d6769d - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:45.552 4076 3051095194792 MainThread DEBUG nova.compute.manager [req-fdd4476a-1c93-4bae-9120-260285d6769d - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:45.585 4076 3051095194792 MainThread INFO nova.compute.manager [req-fdd4476a-1c93-4bae-9120-260285d6769d - - - - -] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:45.585 4076 3051095194440 MainThread DEBUG nova.network.neutron [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Updated VIF entry in instance network info cache for port 7abbc9b3-0ae7-4623-a681-03cc6e640f08. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:45.585 4076 3051095194440 MainThread DEBUG nova.network.neutron [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Updating instance_info_cache with network_info: [{"id": "7abbc9b3-0ae7-4623-a681-03cc6e640f08", "address": "fa:16:3e:fa:3e:0d", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7abbc9b3-0a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:45.601 4076 3051095194440 MainThread DEBUG oslo_concurrency.lockutils [req-2ba148de-7a9c-405c-8fae-b30006e0f217 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-b0e61418-3187-4180-bea6-4e63a7494fbc" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:45.645 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Setting boot order for instance: instance-00000048: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:01:45.707 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:01:45.738 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '749D53A8-8017-47C4-8851-80D35F2C5FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15160), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:45.873 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:45.873 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:45.873 4076 3051095192152 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:45.873 4076 3051095192152 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:45.889 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '749D53A8-8017-47C4-8851-80D35F2C5FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112410), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:45.889 4076 3051095191624 MainThread DEBUG os_win._utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1812279224395752 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:45.904 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-3c04ec42-03b2-48c0-8f0c-363bb635bd71 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:45.904 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-3c04ec42-03b2-48c0-8f0c-363bb635bd71 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:45.940 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-3c04ec42-03b2-48c0-8f0c-363bb635bd71 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:45.958 4076 3051095192152 MainThread INFO nova.compute.manager [req-3c04ec42-03b2-48c0-8f0c-363bb635bd71 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:46.920 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '31078051-57D3-471A-9420-D7F2D34101EE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15369), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:47.051 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:47.051 4076 3051095191976 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:47.051 4076 3051095192152 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:47.051 4076 3051095192152 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:47.051 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '31078051-57D3-471A-9420-D7F2D34101EE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112469), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:47.077 4076 3051095191624 MainThread DEBUG os_win._utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3694632053375244 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:47.087 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-d309a29c-9f00-47d2-b021-962ea4085d4a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:47.087 4076 3051095192152 MainThread DEBUG nova.virt.hyperv.vmops [req-d309a29c-9f00-47d2-b021-962ea4085d4a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:47.139 4076 3051095192152 MainThread DEBUG nova.compute.manager [req-d309a29c-9f00-47d2-b021-962ea4085d4a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:47.159 4076 3051095192152 MainThread INFO nova.compute.manager [req-d309a29c-9f00-47d2-b021-962ea4085d4a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:47.367 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:01:47.367 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:47.367 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:47.367 4076 3051113294136 MainThread DEBUG os_win.utils.compute.vmutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-0000004b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:01:47.367 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-0000004b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:01:47.429 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'E650F851-DB74-4A2A-A1C9-264D6990AE78', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13759), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:47.543 4076 3051113294136 MainThread DEBUG os_win.utils.jobutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': 'E650F851-DB74-4A2A-A1C9-264D6990AE78', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28698), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:01:47.543 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:47.543 4076 3051113294136 MainThread DEBUG os_win.utils.pathutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-0000004b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:01:47.543 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:47.543 4076 3051113294136 MainThread ERROR nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Traceback (most recent call last): 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(*args, **kwargs) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] admin_password, network_info, block_device_info) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(self, *args, **kwds) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] should_plug_vifs=False) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'req_state': req_state}) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] time.sleep(sleep_time) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return f(*args, **kwargs) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return self._wait_for_job(job_path) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] error_desc=err_desc) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During handling of the above exception, another exception occurred: 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Traceback (most recent call last): 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] yield resources 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] accel_info=accel_info) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(*args, **kwargs) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] admin_password, network_info, block_device_info) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(self, *args, **kwds) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] should_plug_vifs=False) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'req_state': req_state}) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] time.sleep(sleep_time) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return f(*args, **kwargs) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return self._wait_for_job(job_path) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] error_desc=err_desc) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.543 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.543 4076 3051113294136 MainThread INFO nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Terminating instance 2021-09-10 17:01:47.543 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:47.543 4076 3051113294136 MainThread INFO nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Got request to destroy instance 2021-09-10 17:01:47.573 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:47.573 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:47.573 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000004b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:47.573 4076 3051113294136 MainThread INFO nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:47.573 4076 3051113294136 MainThread DEBUG nova.compute.claims [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:47.573 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:47.663 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:47.691 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.hostops [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:47.886 4076 3051113294136 MainThread DEBUG nova.compute.provider_tree [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.scheduler.client.report [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:47.918 4076 3051113294136 MainThread ERROR nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Traceback (most recent call last): 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(*args, **kwargs) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] admin_password, network_info, block_device_info) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(self, *args, **kwds) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] should_plug_vifs=False) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'req_state': req_state}) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] time.sleep(sleep_time) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return f(*args, **kwargs) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return self._wait_for_job(job_path) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] error_desc=err_desc) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] During handling of the above exception, another exception occurred: 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Traceback (most recent call last): 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] accel_info=accel_info) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(*args, **kwargs) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] admin_password, network_info, block_device_info) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return function(self, *args, **kwds) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] should_plug_vifs=False) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'req_state': req_state}) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] time.sleep(sleep_time) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self.force_reraise() 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] raise self.value 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return f(*args, **kwargs) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] return self._wait_for_job(job_path) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] error_desc=err_desc) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. 2021-09-10 17:01:47.918 4076 3051113294136 MainThread TRACE nova.compute.manager [instance: 3111054c-9a20-4851-a793-42fbe94a3375] 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.compute.utils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Build of instance 3111054c-9a20-4851-a793-42fbe94a3375 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000004b' failed to start. Not enough memory in the system to start the virtual machine instance-0000004b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004b' failed to start. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Not enough memory in the system to start the virtual machine instance-0000004b with ram size 1024 megabytes. (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) 'instance-0000004b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 41166C1C-9973-467F-A6C9-EF4EE16154B7) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:47.918 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:01:48.097 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '43575732-26CF-4007-BBCE-B1E0127A22C9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14865), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:48.238 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:48.238 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:48.238 4076 3051095194968 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:48.254 4076 3051095194968 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:48.254 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '43575732-26CF-4007-BBCE-B1E0127A22C9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111419), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:48.254 4076 3051095191624 MainThread DEBUG os_win._utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.546513557434082 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:48.274 4076 3051095194968 MainThread DEBUG nova.compute.manager [req-07157147-65b6-46d8-9577-bf24a73aadaa - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:48.274 4076 3051095194968 MainThread DEBUG nova.virt.hyperv.vmops [req-07157147-65b6-46d8-9577-bf24a73aadaa - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:48.301 4076 3051095194968 MainThread DEBUG nova.compute.manager [req-07157147-65b6-46d8-9577-bf24a73aadaa - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:48.348 4076 3051095194968 MainThread INFO nova.compute.manager [req-07157147-65b6-46d8-9577-bf24a73aadaa - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:49.237 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:49.270 4076 3051113294136 MainThread INFO nova.compute.manager [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 3111054c-9a20-4851-a793-42fbe94a3375] Took 1.34 seconds to deallocate network for instance. 2021-09-10 17:01:49.303 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': 'DC5EAD78-31CD-49D0-9AED-0546AB10C6E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15434), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:49.442 4076 3051095194968 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:49.442 4076 3051095194968 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:49.442 4076 3051095194616 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:49.442 4076 3051095194616 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:49.457 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DC5EAD78-31CD-49D0-9AED-0546AB10C6E2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119106), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:49.457 4076 3051095191624 MainThread DEBUG os_win._utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.749589681625366 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:49.476 4076 3051095194616 MainThread DEBUG nova.compute.manager [req-dc5f55a3-f3c6-47fb-92c3-899c40f177b5 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:49.476 4076 3051095194616 MainThread DEBUG nova.virt.hyperv.vmops [req-dc5f55a3-f3c6-47fb-92c3-899c40f177b5 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:49.540 4076 3051095194616 MainThread DEBUG nova.compute.manager [req-dc5f55a3-f3c6-47fb-92c3-899c40f177b5 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:49.585 4076 3051095194616 MainThread INFO nova.compute.manager [req-dc5f55a3-f3c6-47fb-92c3-899c40f177b5 - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:49.657 4076 3051113294136 MainThread INFO nova.scheduler.client.report [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance 3111054c-9a20-4851-a793-42fbe94a3375 2021-09-10 17:01:49.754 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-3e427c4d-772a-4f4c-b7a1-e4cbd186ea13 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "3111054c-9a20-4851-a793-42fbe94a3375" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.922s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:50.056 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-07f66a2c-6f5d-47bf-a64f-842cc6dce48a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Received event network-vif-plugged-7abbc9b3-0ae7-4623-a681-03cc6e640f08 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:50.056 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-07f66a2c-6f5d-47bf-a64f-842cc6dce48a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:50.056 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-07f66a2c-6f5d-47bf-a64f-842cc6dce48a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:50.056 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-07f66a2c-6f5d-47bf-a64f-842cc6dce48a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Processing event network-vif-plugged-7abbc9b3-0ae7-4623-a681-03cc6e640f08 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:01:50.130 4076 3051095192856 MainThread INFO nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Using config drive for instance 2021-09-10 17:01:50.473 4076 3051095192856 MainThread INFO nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Creating config drive at C:\openstack\instances\instance-0000004c\configdrive.iso 2021-09-10 17:01:50.522 4076 3051095192856 MainThread DEBUG oslo_concurrency.processutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp6irsjvwm execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:01:50.660 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '78674371-7D4A-4491-A1E6-0B8FD4288022', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15965), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:50.682 4076 3051095195320 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:50.682 4076 3051095195320 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:50.682 4076 3051095195144 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:50.682 4076 3051095195144 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:50.707 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-5a8f983f-faa8-4410-a9ee-b482b9d6ea8d - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:50.707 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-5a8f983f-faa8-4410-a9ee-b482b9d6ea8d - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:50.747 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-5a8f983f-faa8-4410-a9ee-b482b9d6ea8d - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:50.784 4076 3051095195144 MainThread INFO nova.compute.manager [req-5a8f983f-faa8-4410-a9ee-b482b9d6ea8d - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:50.801 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '78674371-7D4A-4491-A1E6-0B8FD4288022', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111648), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:50.801 4076 3051095191624 MainThread DEBUG os_win._utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.093586206436157 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:50.837 4076 3051095192856 MainThread DEBUG oslo_concurrency.processutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp6irsjvwm" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:01:51.230 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Setting boot order for instance: instance-0000004c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:01:51.269 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:01:51.307 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '2C709A5A-300A-4215-A129-2B9E95484F45', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14880), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:51.446 4076 3051113294136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:51.446 4076 3051113294136 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:51.446 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:51.446 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:51.459 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2C709A5A-300A-4215-A129-2B9E95484F45', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113552), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:51.459 4076 3051095192856 MainThread DEBUG os_win._utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18989968299865723 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:51.459 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-d88c1e94-4472-45fc-be96-880b3798e6fa - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:51.459 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-d88c1e94-4472-45fc-be96-880b3798e6fa - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:51.510 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-d88c1e94-4472-45fc-be96-880b3798e6fa - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:51.538 4076 3051113294136 MainThread INFO nova.compute.manager [req-d88c1e94-4472-45fc-be96-880b3798e6fa - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:51.848 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '1843922A-4AB1-402C-9473-0B1A5C0E3FE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15144), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:51.973 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:51.973 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:51.973 4076 3051113294136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:51.973 4076 3051113294136 MainThread INFO nova.compute.manager [-] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] VM Stopped (Lifecycle Event) 2021-09-10 17:01:51.973 4076 3051095191624 MainThread ERROR os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000048\' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000048\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)instance-000000484C3C35E4-7CE9-4D9F-9646-99AB385FCF4D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1843922A-4AB1-402C-9473-0B1A5C0E3FE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111691), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\nNot enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)\n\n'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D)", 'ErrorSummaryDescription': "'instance-00000048' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000048.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:51.973 4076 3051095191624 MainThread ERROR nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Failed to change vm state of instance-00000048 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. 2021-09-10 17:01:51.973 4076 3051095191624 MainThread INFO nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Got request to destroy instance 2021-09-10 17:01:52.003 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-ce3d2a54-746d-4a9d-a4b9-f0825d928c7a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:52.003 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-ce3d2a54-746d-4a9d-a4b9-f0825d928c7a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:52.050 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-ce3d2a54-746d-4a9d-a4b9-f0825d928c7a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:52.081 4076 3051113294136 MainThread INFO nova.compute.manager [req-ce3d2a54-746d-4a9d-a4b9-f0825d928c7a - - - - -] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:52.473 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '5979AE28-8885-463F-A73C-9EE372F16058', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15112), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:52.598 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:52.598 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:52.598 4076 3051113294136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:52.598 4076 3051113294136 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:52.598 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5979AE28-8885-463F-A73C-9EE372F16058', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113513), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:52.598 4076 3051095192856 MainThread DEBUG os_win._utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3280715942382812 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:52.645 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-d96c0aa9-d406-4cc1-9e76-8d301ca118ee - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:52.645 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-d96c0aa9-d406-4cc1-9e76-8d301ca118ee - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:52.697 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-d96c0aa9-d406-4cc1-9e76-8d301ca118ee - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:52.710 4076 3051113294136 MainThread INFO nova.compute.manager [req-d96c0aa9-d406-4cc1-9e76-8d301ca118ee - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:53.614 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'ED237A27-510B-473E-83EA-23AC2A4122DC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15643), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:53.760 4076 3051113294136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:53.760 4076 3051113294136 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:53.760 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:53.760 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:53.760 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'ED237A27-510B-473E-83EA-23AC2A4122DC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114511), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:53.760 4076 3051095192856 MainThread DEBUG os_win._utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.490072011947632 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:53.795 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-5020b8a2-55a6-4529-80de-86f10535df81 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:53.795 4076 3051113294136 MainThread DEBUG nova.virt.hyperv.vmops [req-5020b8a2-55a6-4529-80de-86f10535df81 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:53.837 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:01:53.837 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:53.837 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:53.837 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-5020b8a2-55a6-4529-80de-86f10535df81 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:53.848 4076 3051095191624 MainThread DEBUG os_win.utils.compute.vmutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Successfully changed vm state of instance-00000048 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:01:53.848 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Successfully changed state of VM instance-00000048 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:01:53.884 4076 3051113294136 MainThread INFO nova.compute.manager [req-5020b8a2-55a6-4529-80de-86f10535df81 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:53.884 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Waiting for WMI job: {'InstanceID': '6150AC02-8342-4528-A9C0-EAF54250BD04', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14310), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:54.023 4076 3051095191624 MainThread DEBUG os_win.utils.jobutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] WMI job succeeded: {'InstanceID': '6150AC02-8342-4528-A9C0-EAF54250BD04', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29273), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:01:54.023 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.037 4076 3051095191624 MainThread DEBUG os_win.utils.pathutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Removing directory: C:\openstack\instances\instance-00000048 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:01:54.037 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:54.037 4076 3051095191624 MainThread ERROR nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Traceback (most recent call last): 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(*args, **kwargs) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] admin_password, network_info, block_device_info) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(self, *args, **kwds) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] should_plug_vifs=False) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'req_state': req_state}) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] time.sleep(sleep_time) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return f(*args, **kwargs) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return self._wait_for_job(job_path) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] error_desc=err_desc) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During handling of the above exception, another exception occurred: 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Traceback (most recent call last): 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] yield resources 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] accel_info=accel_info) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(*args, **kwargs) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] admin_password, network_info, block_device_info) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(self, *args, **kwds) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] should_plug_vifs=False) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'req_state': req_state}) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] time.sleep(sleep_time) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return f(*args, **kwargs) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return self._wait_for_job(job_path) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] error_desc=err_desc) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.037 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.037 4076 3051095191624 MainThread INFO nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Terminating instance 2021-09-10 17:01:54.037 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:54.037 4076 3051095191624 MainThread INFO nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Got request to destroy instance 2021-09-10 17:01:54.057 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:54.057 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.057 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "instance-00000048" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:54.057 4076 3051095191624 MainThread INFO nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:54.066 4076 3051095191624 MainThread DEBUG nova.compute.claims [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:54.066 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.082 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.127 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:01:54.160 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.hostops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:54.191 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.hostops [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:54.412 4076 3051095191624 MainThread DEBUG nova.compute.provider_tree [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:54.412 4076 3051095191624 MainThread DEBUG nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:54.412 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:54.412 4076 3051095191624 MainThread ERROR nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Traceback (most recent call last): 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(*args, **kwargs) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] admin_password, network_info, block_device_info) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(self, *args, **kwds) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] should_plug_vifs=False) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'req_state': req_state}) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] time.sleep(sleep_time) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return f(*args, **kwargs) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return self._wait_for_job(job_path) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] error_desc=err_desc) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] During handling of the above exception, another exception occurred: 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Traceback (most recent call last): 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] accel_info=accel_info) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(*args, **kwargs) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] admin_password, network_info, block_device_info) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return function(self, *args, **kwds) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] should_plug_vifs=False) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'req_state': req_state}) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] time.sleep(sleep_time) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self.force_reraise() 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] raise self.value 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return f(*args, **kwargs) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] return self._wait_for_job(job_path) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] error_desc=err_desc) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. 2021-09-10 17:01:54.412 4076 3051095191624 MainThread TRACE nova.compute.manager [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.compute.utils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Build of instance 9015b51e-d705-4694-8c71-5322dc51aa39 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000048' failed to start. Not enough memory in the system to start the virtual machine instance-00000048. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000048' failed to start. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Not enough memory in the system to start the virtual machine instance-00000048 with ram size 1024 megabytes. (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) 'instance-00000048' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4C3C35E4-7CE9-4D9F-9646-99AB385FCF4D) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:54.440 4076 3051095191624 MainThread DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:01:54.440 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.456 4076 3051095195144 MainThread DEBUG nova.virt.hardware [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:01:54.456 4076 3051095195144 MainThread INFO nova.compute.claims [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Claim successful on node n-h1-808474-1 2021-09-10 17:01:54.551 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.hostops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:54.582 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.hostops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:54.770 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '56DAF236-3FEA-44EE-A3D9-77F74B1A2F99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18825), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:54.770 4076 3051095195144 MainThread DEBUG nova.compute.provider_tree [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:54.798 4076 3051095195144 MainThread DEBUG nova.scheduler.client.report [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:54.798 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:54.798 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:01:54.878 4076 3051095195144 MainThread DEBUG nova.compute.utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:01:54.878 4076 3051095194968 GreenThread-65 DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:01:54.878 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:01:54.894 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:01:54.910 4076 3051095195496 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:54.910 4076 3051095195496 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:54.910 4076 3051095195320 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:54.910 4076 3051095195320 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:54.925 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '56DAF236-3FEA-44EE-A3D9-77F74B1A2F99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116550), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:54.925 4076 3051095192856 MainThread DEBUG os_win._utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6558632850646973 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:54.957 4076 3051095195320 MainThread DEBUG nova.compute.manager [req-2c523c3a-e34e-4cb8-a4c8-574300aff0a8 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:54.957 4076 3051095195320 MainThread DEBUG nova.virt.hyperv.vmops [req-2c523c3a-e34e-4cb8-a4c8-574300aff0a8 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:54.993 4076 3051095195320 MainThread DEBUG nova.compute.manager [req-2c523c3a-e34e-4cb8-a4c8-574300aff0a8 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:55.049 4076 3051095195320 MainThread INFO nova.compute.manager [req-2c523c3a-e34e-4cb8-a4c8-574300aff0a8 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:55.151 4076 3051095194968 GreenThread-65 DEBUG nova.policy [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:01:55.151 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:01:55.170 4076 3051095195144 MainThread INFO nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Spawning new instance 2021-09-10 17:01:55.170 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:55.170 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:55.170 4076 3051095195144 MainThread DEBUG nova.block_device [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:01:55.170 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:55.170 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:55.190 4076 3051095195144 MainThread DEBUG os_win.utils.pathutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-0000004e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:01:55.190 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000004e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:01:55.210 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Resizing VHD C:\openstack\instances\instance-0000004e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:01:55.956 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '9F247826-130C-4EB5-9AB6-CC0F4E6B8ECF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14632), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:56.021 4076 3051095191624 MainThread DEBUG nova.network.neutron [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:56.070 4076 3051095191624 MainThread INFO nova.compute.manager [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] [instance: 9015b51e-d705-4694-8c71-5322dc51aa39] Took 1.62 seconds to deallocate network for instance. 2021-09-10 17:01:56.088 4076 3051095193384 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:56.088 4076 3051095193384 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:56.088 4076 3051095195320 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:56.088 4076 3051095195320 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:56.088 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Successfully created port: 557c75cf-cb9a-4924-b06c-e71017177bf0 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:01:56.113 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9F247826-130C-4EB5-9AB6-CC0F4E6B8ECF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111920), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:56.113 4076 3051095192856 MainThread DEBUG os_win._utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.843669176101685 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:01:56.113 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-fc8cf66f-8ae4-4746-adf2-03ec8798c838 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:56.113 4076 3051095193384 MainThread DEBUG nova.virt.hyperv.vmops [req-fc8cf66f-8ae4-4746-adf2-03ec8798c838 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:56.159 4076 3051095193384 MainThread DEBUG nova.compute.manager [req-fc8cf66f-8ae4-4746-adf2-03ec8798c838 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:56.206 4076 3051095193384 MainThread INFO nova.compute.manager [req-fc8cf66f-8ae4-4746-adf2-03ec8798c838 - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:56.285 4076 3051095191624 MainThread INFO nova.scheduler.client.report [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Deleted allocations for instance 9015b51e-d705-4694-8c71-5322dc51aa39 2021-09-10 17:01:56.340 4076 3051095191624 MainThread DEBUG oslo_concurrency.lockutils [req-37727a18-294f-44c4-8cdc-cc3dc02a2428 0574bfa519834ae2b8ce81f3a560b957 58b055dc3ac4474788971e8148bec63e - default default] Lock "9015b51e-d705-4694-8c71-5322dc51aa39" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 20.875s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:57.129 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'F238EB08-12C5-42C4-AF61-4207CAF52A0D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16998), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:57.254 4076 3051095192856 MainThread ERROR os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004c\' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)instance-0000004cC387DA62-9BCF-418E-A06E-3D75E796400A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F238EB08-12C5-42C4-AF61-4207CAF52A0D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114024), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 1, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 1, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\nNot enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)\n\n'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A)", 'ErrorSummaryDescription': "'instance-0000004c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:01:57.285 4076 3051095192856 MainThread ERROR nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Failed to change vm state of instance-0000004c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. 2021-09-10 17:01:57.285 4076 3051095192856 MainThread INFO nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Got request to destroy instance 2021-09-10 17:01:57.285 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:57.285 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:57.285 4076 3051095191624 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:01:57.285 4076 3051095191624 MainThread INFO nova.compute.manager [-] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] VM Stopped (Lifecycle Event) 2021-09-10 17:01:57.285 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Successfully updated port: 557c75cf-cb9a-4924-b06c-e71017177bf0 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:01:57.317 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-ec41e18f-cec7-4928-b9de-be736bb8d19a - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:01:57.317 4076 3051095191624 MainThread DEBUG nova.virt.hyperv.vmops [req-ec41e18f-cec7-4928-b9de-be736bb8d19a - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:01:57.317 4076 3051095194968 GreenThread-65 DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-22ce4089-0acb-4f53-bb15-501df1d949ed" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:57.317 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:01:57.363 4076 3051095191624 MainThread DEBUG nova.compute.manager [req-ec41e18f-cec7-4928-b9de-be736bb8d19a - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:01:57.395 4076 3051095191624 MainThread INFO nova.compute.manager [req-ec41e18f-cec7-4928-b9de-be736bb8d19a - - - - -] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:01:57.395 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:01:57.680 4076 3051095194616 MainThread DEBUG nova.compute.manager [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Received event network-changed-557c75cf-cb9a-4924-b06c-e71017177bf0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:01:57.680 4076 3051095194616 MainThread DEBUG nova.compute.manager [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Refreshing instance network info cache due to event network-changed-557c75cf-cb9a-4924-b06c-e71017177bf0. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:01:57.913 4076 3051095194968 GreenThread-65 DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Updating instance_info_cache with network_info: [{"id": "557c75cf-cb9a-4924-b06c-e71017177bf0", "address": "fa:16:3e:06:6c:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap557c75cf-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:57.935 4076 3051095194968 GreenThread-65 DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-22ce4089-0acb-4f53-bb15-501df1d949ed" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:57.935 4076 3051095194968 GreenThread-65 DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Instance network_info: |[{"id": "557c75cf-cb9a-4924-b06c-e71017177bf0", "address": "fa:16:3e:06:6c:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap557c75cf-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:01:57.935 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Preparing to wait for external event network-vif-plugged-557c75cf-cb9a-4924-b06c-e71017177bf0 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:01:57.935 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:57.935 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:57.935 4076 3051095195144 MainThread DEBUG os_win.utils.compute.vmutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-0000004e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:01:57.941 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-22ce4089-0acb-4f53-bb15-501df1d949ed" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:01:57.941 4076 3051095194616 MainThread DEBUG nova.network.neutron [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Refreshing network info cache for port 557c75cf-cb9a-4924-b06c-e71017177bf0 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:01:58.847 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:01:58.966 4076 3051095194616 MainThread DEBUG nova.network.neutron [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Updated VIF entry in instance network info cache for port 557c75cf-cb9a-4924-b06c-e71017177bf0. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:01:58.966 4076 3051095194616 MainThread DEBUG nova.network.neutron [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Updating instance_info_cache with network_info: [{"id": "557c75cf-cb9a-4924-b06c-e71017177bf0", "address": "fa:16:3e:06:6c:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap557c75cf-cb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:01:58.989 4076 3051095194616 MainThread DEBUG oslo_concurrency.lockutils [req-00e73fd4-eb35-42f9-9b72-5315abe3a8a5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-22ce4089-0acb-4f53-bb15-501df1d949ed" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:01:59.160 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:01:59.160 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:59.160 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:59.176 4076 3051095192856 MainThread DEBUG os_win.utils.compute.vmutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed vm state of instance-0000004c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:01:59.176 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed state of VM instance-0000004c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:01:59.234 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '2846EF0B-72AB-44FD-8FEC-E0A6FEB91C1B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14022), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:01:59.358 4076 3051095192856 MainThread DEBUG os_win.utils.jobutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job succeeded: {'InstanceID': '2846EF0B-72AB-44FD-8FEC-E0A6FEB91C1B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28551), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:01:59.358 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:59.358 4076 3051095192856 MainThread DEBUG os_win.utils.pathutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Removing directory: C:\openstack\instances\instance-0000004c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:01:59.358 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:59.358 4076 3051095192856 MainThread ERROR nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Traceback (most recent call last): 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(*args, **kwargs) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] admin_password, network_info, block_device_info) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(self, *args, **kwds) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] should_plug_vifs=False) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'req_state': req_state}) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] time.sleep(sleep_time) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return f(*args, **kwargs) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return self._wait_for_job(job_path) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] error_desc=err_desc) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During handling of the above exception, another exception occurred: 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Traceback (most recent call last): 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] yield resources 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] accel_info=accel_info) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(*args, **kwargs) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] admin_password, network_info, block_device_info) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(self, *args, **kwds) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] should_plug_vifs=False) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'req_state': req_state}) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] time.sleep(sleep_time) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return f(*args, **kwargs) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return self._wait_for_job(job_path) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] error_desc=err_desc) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.358 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.369 4076 3051095192856 MainThread INFO nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Terminating instance 2021-09-10 17:01:59.369 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:01:59.369 4076 3051095192856 MainThread INFO nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Got request to destroy instance 2021-09-10 17:01:59.380 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:01:59.380 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:59.380 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-0000004c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:59.380 4076 3051095192856 MainThread INFO nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 17:01:59.380 4076 3051095192856 MainThread DEBUG nova.compute.claims [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:01:59.380 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:01:59.550 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.hostops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:01:59.566 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.hostops [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:01:59.770 4076 3051095192856 MainThread DEBUG nova.compute.provider_tree [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:01:59.786 4076 3051095192856 MainThread DEBUG nova.scheduler.client.report [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:01:59.786 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:01:59.786 4076 3051095192856 MainThread ERROR nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Traceback (most recent call last): 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(*args, **kwargs) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] admin_password, network_info, block_device_info) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(self, *args, **kwds) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] should_plug_vifs=False) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'req_state': req_state}) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] time.sleep(sleep_time) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return f(*args, **kwargs) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return self._wait_for_job(job_path) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] error_desc=err_desc) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] During handling of the above exception, another exception occurred: 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Traceback (most recent call last): 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] accel_info=accel_info) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(*args, **kwargs) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] admin_password, network_info, block_device_info) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return function(self, *args, **kwds) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] should_plug_vifs=False) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'req_state': req_state}) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] time.sleep(sleep_time) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self.force_reraise() 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] raise self.value 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return f(*args, **kwargs) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] return self._wait_for_job(job_path) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] error_desc=err_desc) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. 2021-09-10 17:01:59.786 4076 3051095192856 MainThread TRACE nova.compute.manager [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] 2021-09-10 17:01:59.786 4076 3051095192856 MainThread DEBUG nova.compute.utils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:01:59.786 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Build of instance b0e61418-3187-4180-bea6-4e63a7494fbc was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000004c' failed to start. Not enough memory in the system to start the virtual machine instance-0000004c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004c' failed to start. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Not enough memory in the system to start the virtual machine instance-0000004c with ram size 1024 megabytes. (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) 'instance-0000004c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C387DA62-9BCF-418E-A06E-3D75E796400A) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:01:59.801 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:01:59.801 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:01:59.801 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:01:59.801 4076 3051095192856 MainThread DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:01.146 4076 3051095192856 MainThread DEBUG nova.network.neutron [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:01.175 4076 3051095192856 MainThread INFO nova.compute.manager [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: b0e61418-3187-4180-bea6-4e63a7494fbc] Took 1.38 seconds to deallocate network for instance. 2021-09-10 17:02:01.418 4076 3051095192856 MainThread INFO nova.scheduler.client.report [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Deleted allocations for instance b0e61418-3187-4180-bea6-4e63a7494fbc 2021-09-10 17:02:01.473 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-25254cbc-85c4-48a6-b99a-b205c8bdf5dc b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "b0e61418-3187-4180-bea6-4e63a7494fbc" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 23.641s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:05.223 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-663e9bc4-65e8-4582-826e-af47d478c166 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Received event network-vif-plugged-557c75cf-cb9a-4924-b06c-e71017177bf0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:05.223 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-663e9bc4-65e8-4582-826e-af47d478c166 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:05.223 4076 3051095192856 MainThread DEBUG oslo_concurrency.lockutils [req-663e9bc4-65e8-4582-826e-af47d478c166 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:05.223 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-663e9bc4-65e8-4582-826e-af47d478c166 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Processing event network-vif-plugged-557c75cf-cb9a-4924-b06c-e71017177bf0 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:05.253 4076 3051095195144 MainThread INFO nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Using config drive for instance 2021-09-10 17:02:05.551 4076 3051095195144 MainThread INFO nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Creating config drive at C:\openstack\instances\instance-0000004e\configdrive.iso 2021-09-10 17:02:05.599 4076 3051095195144 MainThread DEBUG oslo_concurrency.processutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl899lrmh execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:05.899 4076 3051095195144 MainThread DEBUG oslo_concurrency.processutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl899lrmh" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:06.290 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-0000004e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:06.341 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:06.364 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '5F44A3DD-135D-48EA-BBF3-3FEE15930FB3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16341), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:06.504 4076 3051095192856 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:06.504 4076 3051095192856 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:06.504 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:06.504 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:06.504 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5F44A3DD-135D-48EA-BBF3-3FEE15930FB3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111109), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:06.504 4076 3051095195144 MainThread DEBUG os_win._utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16304492950439453 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:06.530 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-bb2c833d-a250-4b86-80be-fe91291e3d42 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:06.530 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-bb2c833d-a250-4b86-80be-fe91291e3d42 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:06.582 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-bb2c833d-a250-4b86-80be-fe91291e3d42 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:06.614 4076 3051095192856 MainThread INFO nova.compute.manager [req-bb2c833d-a250-4b86-80be-fe91291e3d42 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:07.536 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '30FDE483-C2AD-456C-9B6B-DD4DA62F6477', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15441), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:07.659 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:07.659 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:07.659 4076 3051095192856 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:07.659 4076 3051095192856 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:07.690 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '30FDE483-C2AD-456C-9B6B-DD4DA62F6477', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115100), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:07.690 4076 3051095195144 MainThread DEBUG os_win._utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3489699363708496 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:07.690 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-44c42cd9-7fef-4396-acd7-4725a0d73f92 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:07.690 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-44c42cd9-7fef-4396-acd7-4725a0d73f92 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:07.721 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-44c42cd9-7fef-4396-acd7-4725a0d73f92 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:07.784 4076 3051095192856 MainThread INFO nova.compute.manager [req-44c42cd9-7fef-4396-acd7-4725a0d73f92 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:08.739 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'B398BFE8-6B95-44D3-A229-CE9AF7032F7B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15673), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:08.864 4076 3051095192856 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:08.864 4076 3051095192856 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:08.864 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:08.864 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:08.864 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B398BFE8-6B95-44D3-A229-CE9AF7032F7B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113703), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:08.864 4076 3051095195144 MainThread DEBUG os_win._utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5235960483551025 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:08.893 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-b2294a8e-5b0a-479f-b31c-a19e70904313 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:08.893 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-b2294a8e-5b0a-479f-b31c-a19e70904313 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:08.940 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-b2294a8e-5b0a-479f-b31c-a19e70904313 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:08.973 4076 3051095192856 MainThread INFO nova.compute.manager [req-b2294a8e-5b0a-479f-b31c-a19e70904313 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:09.899 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '8388D1D9-4BE3-4283-BBA7-95F0A510B9E0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15301), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:10.024 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:10.024 4076 3051095193384 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:10.024 4076 3051095192856 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:10.024 4076 3051095192856 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:10.046 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8388D1D9-4BE3-4283-BBA7-95F0A510B9E0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110610), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:10.046 4076 3051095195144 MainThread DEBUG os_win._utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.705191135406494 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:10.081 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-c8e85d32-13d7-4699-8cb3-55e0c06ff123 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:10.081 4076 3051095192856 MainThread DEBUG nova.virt.hyperv.vmops [req-c8e85d32-13d7-4699-8cb3-55e0c06ff123 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:10.097 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:10.129 4076 3051095192856 MainThread DEBUG nova.compute.manager [req-c8e85d32-13d7-4699-8cb3-55e0c06ff123 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:10.144 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:10.175 4076 3051095192856 MainThread INFO nova.compute.manager [req-c8e85d32-13d7-4699-8cb3-55e0c06ff123 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:10.291 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:10.291 4076 3051114311928 MainThread DEBUG nova.virt.hardware [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:10.291 4076 3051114311928 MainThread INFO nova.compute.claims [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Claim successful on node n-h1-808474-1 2021-09-10 17:02:10.473 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.hostops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:10.508 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.hostops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:10.691 4076 3051114311928 MainThread DEBUG nova.compute.provider_tree [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:10.734 4076 3051114311928 MainThread DEBUG nova.scheduler.client.report [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:10.734 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.454s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:10.734 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:10.868 4076 3051114311928 MainThread DEBUG nova.compute.utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:10.868 4076 3051095192856 GreenThread-66 DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:10.868 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:10.895 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:11.082 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '3486DFFE-7B90-4377-92E9-7510C6A22BEA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15256), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:11.160 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:11.160 4076 3051114311928 MainThread INFO nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Spawning new instance 2021-09-10 17:02:11.178 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:11.178 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:11.178 4076 3051114311928 MainThread DEBUG nova.block_device [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:11.178 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:11.178 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:11.190 4076 3051114311928 MainThread DEBUG os_win.utils.pathutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Creating directory: C:\openstack\instances\instance-0000004f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:11.190 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000004f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:11.223 4076 3051095192856 GreenThread-66 DEBUG nova.policy [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'a57565d91e984026bfef16cbf8b6f2bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57e8809b286f4c5aad08911c6a289582', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:11.223 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:11.223 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:11.223 4076 3051114312104 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:11.223 4076 3051114312104 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:11.223 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3486DFFE-7B90-4377-92E9-7510C6A22BEA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110428), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:11.223 4076 3051095195144 MainThread DEBUG os_win._utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.881839752197266 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:11.223 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Resizing VHD C:\openstack\instances\instance-0000004f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:11.252 4076 3051114312104 MainThread DEBUG nova.compute.manager [req-094dae45-b33a-496e-975e-f5518ca8049b - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:11.252 4076 3051114312104 MainThread DEBUG nova.virt.hyperv.vmops [req-094dae45-b33a-496e-975e-f5518ca8049b - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:11.317 4076 3051114312104 MainThread DEBUG nova.compute.manager [req-094dae45-b33a-496e-975e-f5518ca8049b - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:11.348 4076 3051114312104 MainThread INFO nova.compute.manager [req-094dae45-b33a-496e-975e-f5518ca8049b - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:12.239 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '64C2B49F-4F78-4C21-91B1-644C228BAC51', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15720), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:12.365 4076 3051114312104 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:12.365 4076 3051114312104 MainThread INFO nova.compute.manager [-] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] VM Stopped (Lifecycle Event) 2021-09-10 17:02:12.389 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:12.389 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:12.389 4076 3051095195144 MainThread ERROR os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004e\' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE312030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE310243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)instance-0000004eE765720B-AB66-4563-9FFB-2B94C5DECAE3%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '64C2B49F-4F78-4C21-91B1-644C228BAC51', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111152), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\nNot enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)\n\n'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3)", 'ErrorSummaryDescription': "'instance-0000004e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:12.395 4076 3051095195144 MainThread ERROR nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-0000004e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. 2021-09-10 17:02:12.395 4076 3051095195144 MainThread INFO nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Got request to destroy instance 2021-09-10 17:02:12.425 4076 3051114312104 MainThread DEBUG nova.compute.manager [req-38611b89-9de1-42de-a68c-4faa5c5ebb93 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:12.425 4076 3051114312104 MainThread DEBUG nova.virt.hyperv.vmops [req-38611b89-9de1-42de-a68c-4faa5c5ebb93 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:12.473 4076 3051114312104 MainThread DEBUG nova.compute.manager [req-38611b89-9de1-42de-a68c-4faa5c5ebb93 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:12.513 4076 3051114312104 MainThread INFO nova.compute.manager [req-38611b89-9de1-42de-a68c-4faa5c5ebb93 - - - - -] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:12.644 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:12.644 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:12.679 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Successfully created port: 37edf4d4-e554-46d2-a421-726eba645ac3 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:12.833 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:12.833 4076 3051114312280 MainThread DEBUG nova.virt.hardware [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:12.833 4076 3051114312280 MainThread INFO nova.compute.claims [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Claim successful on node n-h1-808474-1 2021-09-10 17:02:13.117 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.hostops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:13.117 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.hostops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:13.336 4076 3051114312280 MainThread DEBUG nova.compute.provider_tree [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:13.363 4076 3051114312280 MainThread DEBUG nova.scheduler.client.report [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:13.363 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.532s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:13.363 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:13.460 4076 3051114312280 MainThread DEBUG nova.compute.utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:13.460 4076 3051114312104 GreenThread-67 DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:13.460 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:13.502 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:13.725 4076 3051114312104 GreenThread-67 DEBUG nova.policy [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0063b05acec342499daf8129dc0ca365', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d6026a33839407f9f68cb295fbb6928', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:13.818 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:13.818 4076 3051114312280 MainThread INFO nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Spawning new instance 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG nova.block_device [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG os_win.utils.pathutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Creating directory: C:\openstack\instances\instance-00000050 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:13.832 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000050\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:13.862 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Resizing VHD C:\openstack\instances\instance-00000050\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:14.252 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:02:14.276 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:14.276 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:14.276 4076 3051095195144 MainThread DEBUG os_win.utils.compute.vmutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-0000004e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:02:14.276 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-0000004e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:02:14.323 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '24B061B7-B930-4C6F-BC36-6077241973D7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14443), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:14.425 4076 3051095195144 MainThread DEBUG os_win.utils.jobutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': '24B061B7-B930-4C6F-BC36-6077241973D7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28795), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:02:14.425 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:14.425 4076 3051095195144 MainThread DEBUG os_win.utils.pathutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-0000004e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:02:14.425 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:14.425 4076 3051095195144 MainThread ERROR nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Traceback (most recent call last): 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(*args, **kwargs) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] admin_password, network_info, block_device_info) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(self, *args, **kwds) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] should_plug_vifs=False) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'req_state': req_state}) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] time.sleep(sleep_time) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return f(*args, **kwargs) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return self._wait_for_job(job_path) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] error_desc=err_desc) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During handling of the above exception, another exception occurred: 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Traceback (most recent call last): 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] yield resources 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] accel_info=accel_info) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(*args, **kwargs) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] admin_password, network_info, block_device_info) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(self, *args, **kwds) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] should_plug_vifs=False) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'req_state': req_state}) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] time.sleep(sleep_time) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return f(*args, **kwargs) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return self._wait_for_job(job_path) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] error_desc=err_desc) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.425 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.447 4076 3051095195144 MainThread INFO nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Terminating instance 2021-09-10 17:02:14.447 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:02:14.447 4076 3051095195144 MainThread INFO nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Got request to destroy instance 2021-09-10 17:02:14.447 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.vmops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:02:14.469 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:14.469 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000004e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:14.469 4076 3051095195144 MainThread INFO nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:02:14.473 4076 3051095195144 MainThread DEBUG nova.compute.claims [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:02:14.473 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:14.590 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.hostops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:14.610 4076 3051095195144 MainThread DEBUG nova.virt.hyperv.hostops [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:14.800 4076 3051095195144 MainThread DEBUG nova.compute.provider_tree [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:14.817 4076 3051095195144 MainThread DEBUG nova.scheduler.client.report [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:14.817 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:14.817 4076 3051095195144 MainThread ERROR nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Traceback (most recent call last): 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(*args, **kwargs) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] admin_password, network_info, block_device_info) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(self, *args, **kwds) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] should_plug_vifs=False) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'req_state': req_state}) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] time.sleep(sleep_time) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return f(*args, **kwargs) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return self._wait_for_job(job_path) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] error_desc=err_desc) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] During handling of the above exception, another exception occurred: 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Traceback (most recent call last): 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] accel_info=accel_info) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(*args, **kwargs) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] admin_password, network_info, block_device_info) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return function(self, *args, **kwds) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] should_plug_vifs=False) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'req_state': req_state}) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] time.sleep(sleep_time) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self.force_reraise() 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] raise self.value 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return f(*args, **kwargs) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] return self._wait_for_job(job_path) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] error_desc=err_desc) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. 2021-09-10 17:02:14.817 4076 3051095195144 MainThread TRACE nova.compute.manager [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.compute.utils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Build of instance 22ce4089-0acb-4f53-bb15-501df1d949ed was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000004e' failed to start. Not enough memory in the system to start the virtual machine instance-0000004e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004e' failed to start. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Not enough memory in the system to start the virtual machine instance-0000004e with ram size 1024 megabytes. (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) 'instance-0000004e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E765720B-AB66-4563-9FFB-2B94C5DECAE3) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:02:14.832 4076 3051095195144 MainThread DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:15.019 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Successfully updated port: 37edf4d4-e554-46d2-a421-726eba645ac3 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:15.067 4076 3051095192856 GreenThread-66 DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Acquired lock "refresh_cache-30a2e0f5-74d7-453d-ad6a-628ef91e6d39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:15.067 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:15.596 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:15.695 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:15.721 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:15.784 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:15.833 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:15.925 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:15.941 4076 3051114314216 MainThread DEBUG nova.virt.hardware [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:15.941 4076 3051114314216 MainThread INFO nova.compute.claims [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Claim successful on node n-h1-808474-1 2021-09-10 17:02:16.083 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:16.098 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:16.299 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Successfully created port: 5e9b878b-5ea4-4417-8d82-1b3ad6490c23 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:16.300 4076 3051114314216 MainThread DEBUG nova.compute.provider_tree [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:16.328 4076 3051114314216 MainThread DEBUG nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:16.328 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:16.332 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:16.332 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:16.332 4076 3051114314392 MainThread DEBUG nova.virt.hardware [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:16.332 4076 3051114314392 MainThread INFO nova.compute.claims [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Claim successful on node n-h1-808474-1 2021-09-10 17:02:16.377 4076 3051095192856 GreenThread-66 DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Updating instance_info_cache with network_info: [{"id": "37edf4d4-e554-46d2-a421-726eba645ac3", "address": "fa:16:3e:65:7e:b0", "network": {"id": "a63611d6-d78b-4c79-b9d9-a82ebe7f3184", "bridge": null, "label": "tempest-ServersNegativeTestMultiTenantJSON-1395225850-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "57e8809b286f4c5aad08911c6a289582", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37edf4d4-e5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:16.618 4076 3051095192856 GreenThread-66 DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Releasing lock "refresh_cache-30a2e0f5-74d7-453d-ad6a-628ef91e6d39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:16.618 4076 3051095192856 GreenThread-66 DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Instance network_info: |[{"id": "37edf4d4-e554-46d2-a421-726eba645ac3", "address": "fa:16:3e:65:7e:b0", "network": {"id": "a63611d6-d78b-4c79-b9d9-a82ebe7f3184", "bridge": null, "label": "tempest-ServersNegativeTestMultiTenantJSON-1395225850-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "57e8809b286f4c5aad08911c6a289582", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37edf4d4-e5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:16.618 4076 3051114314216 MainThread DEBUG nova.compute.utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:16.618 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Preparing to wait for external event network-vif-plugged-37edf4d4-e554-46d2-a421-726eba645ac3 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:16.618 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:16.643 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:16.643 4076 3051114311928 MainThread DEBUG os_win.utils.compute.vmutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Creating VM instance-0000004f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:16.643 4076 3051113294136 GreenThread-68 DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:16.643 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:16.661 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:16.690 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:16.758 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:16.856 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:16.864 4076 3051113294136 GreenThread-68 DEBUG nova.policy [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b207fe385eb647a4acf0331024dcf245', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '00a75e521b5f4d89ab74f2701a36da12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:16.864 4076 3051114314216 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Spawning new instance 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG nova.block_device [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG os_win.utils.pathutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating directory: C:\openstack\instances\instance-00000052 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:16.881 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000052\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:16.931 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Resizing VHD C:\openstack\instances\instance-00000052\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:16.973 4076 3051114314392 MainThread DEBUG nova.compute.provider_tree [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:16.988 4076 3051114314392 MainThread DEBUG nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:16.988 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.657s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:16.988 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:17.081 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Received event network-changed-37edf4d4-e554-46d2-a421-726eba645ac3 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:17.081 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Refreshing instance network info cache due to event network-changed-37edf4d4-e554-46d2-a421-726eba645ac3. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:17.081 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-30a2e0f5-74d7-453d-ad6a-628ef91e6d39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:17.081 4076 3051113292024 MainThread DEBUG nova.network.neutron [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Refreshing network info cache for port 37edf4d4-e554-46d2-a421-726eba645ac3 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:17.160 4076 3051114314392 MainThread DEBUG nova.compute.utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:17.160 4076 3051113294488 GreenThread-69 DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:17.160 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:17.183 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:17.437 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:17.441 4076 3051114314392 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Spawning new instance 2021-09-10 17:02:17.466 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:17.466 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:17.466 4076 3051114314392 MainThread DEBUG nova.block_device [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:17.466 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:17.466 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:17.490 4076 3051114314392 MainThread DEBUG os_win.utils.pathutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating directory: C:\openstack\instances\instance-00000053 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:17.490 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000053\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:17.518 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Resizing VHD C:\openstack\instances\instance-00000053\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:17.518 4076 3051113294488 GreenThread-69 DEBUG nova.policy [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b207fe385eb647a4acf0331024dcf245', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '00a75e521b5f4d89ab74f2701a36da12', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:17.518 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:17.893 4076 3051095195144 MainThread DEBUG nova.network.neutron [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:17.924 4076 3051095195144 MainThread INFO nova.compute.manager [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 22ce4089-0acb-4f53-bb15-501df1d949ed] Took 3.09 seconds to deallocate network for instance. 2021-09-10 17:02:18.142 4076 3051095195144 MainThread INFO nova.scheduler.client.report [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance 22ce4089-0acb-4f53-bb15-501df1d949ed 2021-09-10 17:02:18.195 4076 3051095195144 MainThread DEBUG oslo_concurrency.lockutils [req-18c9d1a8-1554-452f-91be-fe40bd4f9fb5 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "22ce4089-0acb-4f53-bb15-501df1d949ed" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.110s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:18.240 4076 3051113292024 MainThread DEBUG nova.network.neutron [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Updated VIF entry in instance network info cache for port 37edf4d4-e554-46d2-a421-726eba645ac3. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:18.240 4076 3051113292024 MainThread DEBUG nova.network.neutron [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Updating instance_info_cache with network_info: [{"id": "37edf4d4-e554-46d2-a421-726eba645ac3", "address": "fa:16:3e:65:7e:b0", "network": {"id": "a63611d6-d78b-4c79-b9d9-a82ebe7f3184", "bridge": null, "label": "tempest-ServersNegativeTestMultiTenantJSON-1395225850-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "57e8809b286f4c5aad08911c6a289582", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap37edf4d4-e5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:18.269 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [req-e7ecf614-90c0-4474-952d-1d670b8834eb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-30a2e0f5-74d7-453d-ad6a-628ef91e6d39" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:18.911 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Successfully created port: c85f831d-3308-4358-afc7-f73a1171d746 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:20.268 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Successfully updated port: 5e9b878b-5ea4-4417-8d82-1b3ad6490c23 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:20.287 4076 3051114312104 GreenThread-67 DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Acquired lock "refresh_cache-8001e033-b435-408f-a338-cd26132a7cc1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:20.287 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:20.708 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:20.834 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Received event network-changed-5e9b878b-5ea4-4417-8d82-1b3ad6490c23 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:20.834 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Refreshing instance network info cache due to event network-changed-5e9b878b-5ea4-4417-8d82-1b3ad6490c23. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:21.080 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Successfully updated port: c85f831d-3308-4358-afc7-f73a1171d746 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:21.086 4076 3051113294136 GreenThread-68 DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Acquired lock "refresh_cache-84fb0901-77a9-482c-9355-34416b3289cb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:21.086 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:21.155 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:21.228 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Successfully created port: b166f44b-43d3-4457-b778-4531693659ec _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:21.487 4076 3051113294136 GreenThread-68 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Updating instance_info_cache with network_info: [{"id": "c85f831d-3308-4358-afc7-f73a1171d746", "address": "fa:16:3e:de:dc:11", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc85f831d-33", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:21.505 4076 3051113294136 GreenThread-68 DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Releasing lock "refresh_cache-84fb0901-77a9-482c-9355-34416b3289cb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:21.505 4076 3051113294136 GreenThread-68 DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Instance network_info: |[{"id": "c85f831d-3308-4358-afc7-f73a1171d746", "address": "fa:16:3e:de:dc:11", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc85f831d-33", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:21.505 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Preparing to wait for external event network-vif-plugged-c85f831d-3308-4358-afc7-f73a1171d746 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:21.505 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:21.505 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:21.505 4076 3051114314216 MainThread DEBUG os_win.utils.compute.vmutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating VM instance-00000052 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:22.381 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:22.661 4076 3051114312104 GreenThread-67 DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Updating instance_info_cache with network_info: [{"id": "5e9b878b-5ea4-4417-8d82-1b3ad6490c23", "address": "fa:16:3e:65:9d:ec", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5e9b878b-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:22.688 4076 3051114312104 GreenThread-67 DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Releasing lock "refresh_cache-8001e033-b435-408f-a338-cd26132a7cc1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:22.688 4076 3051114312104 GreenThread-67 DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Instance network_info: |[{"id": "5e9b878b-5ea4-4417-8d82-1b3ad6490c23", "address": "fa:16:3e:65:9d:ec", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5e9b878b-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:22.688 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-8001e033-b435-408f-a338-cd26132a7cc1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:22.688 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Refreshing network info cache for port 5e9b878b-5ea4-4417-8d82-1b3ad6490c23 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:22.696 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Preparing to wait for external event network-vif-plugged-5e9b878b-5ea4-4417-8d82-1b3ad6490c23 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:22.696 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:22.696 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:22.696 4076 3051114312280 MainThread DEBUG os_win.utils.compute.vmutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Creating VM instance-00000050 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:22.909 4076 3051113292552 MainThread DEBUG nova.compute.manager [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Received event network-changed-c85f831d-3308-4358-afc7-f73a1171d746 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:22.909 4076 3051113292552 MainThread DEBUG nova.compute.manager [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Refreshing instance network info cache due to event network-changed-c85f831d-3308-4358-afc7-f73a1171d746. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:22.909 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-84fb0901-77a9-482c-9355-34416b3289cb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:22.909 4076 3051113292552 MainThread DEBUG nova.network.neutron [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Refreshing network info cache for port c85f831d-3308-4358-afc7-f73a1171d746 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:22.974 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-bb693c3c-8b3b-47c0-ac00-fae4a6ab85d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Received event network-vif-plugged-37edf4d4-e554-46d2-a421-726eba645ac3 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:22.974 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-bb693c3c-8b3b-47c0-ac00-fae4a6ab85d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:22.974 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-bb693c3c-8b3b-47c0-ac00-fae4a6ab85d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:22.974 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-bb693c3c-8b3b-47c0-ac00-fae4a6ab85d3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Processing event network-vif-plugged-37edf4d4-e554-46d2-a421-726eba645ac3 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:23.014 4076 3051114311928 MainThread INFO nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Using config drive for instance 2021-09-10 17:02:23.361 4076 3051114311928 MainThread INFO nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Creating config drive at C:\openstack\instances\instance-0000004f\configdrive.iso 2021-09-10 17:02:23.394 4076 3051114311928 MainThread DEBUG oslo_concurrency.processutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3maatpf1 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:23.710 4076 3051114311928 MainThread DEBUG oslo_concurrency.processutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000004f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3maatpf1" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:23.785 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:24.130 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Setting boot order for instance: instance-0000004f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:24.178 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:24.206 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': '1DD3AFB1-D8F4-4386-933F-0EE4A93F5DA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18671), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:24.346 4076 3051113292024 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:24.346 4076 3051113292024 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:24.346 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:24.346 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:24.346 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1DD3AFB1-D8F4-4386-933F-0EE4A93F5DA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112580), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:24.346 4076 3051114311928 MainThread DEBUG os_win._utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1684260368347168 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:24.382 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-63057477-ce4a-48fc-8269-fcd924652e4f - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:24.394 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-63057477-ce4a-48fc-8269-fcd924652e4f - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:24.445 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-63057477-ce4a-48fc-8269-fcd924652e4f - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:24.472 4076 3051113292024 MainThread INFO nova.compute.manager [req-63057477-ce4a-48fc-8269-fcd924652e4f - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:25.096 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Successfully updated port: b166f44b-43d3-4457-b778-4531693659ec _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:25.143 4076 3051113294488 GreenThread-69 DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Acquired lock "refresh_cache-02a3f851-fc39-4889-aaff-e5a3385f19c0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:25.143 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:25.162 4076 3051113292552 MainThread DEBUG nova.network.neutron [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Updated VIF entry in instance network info cache for port c85f831d-3308-4358-afc7-f73a1171d746. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:25.162 4076 3051113292552 MainThread DEBUG nova.network.neutron [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Updating instance_info_cache with network_info: [{"id": "c85f831d-3308-4358-afc7-f73a1171d746", "address": "fa:16:3e:de:dc:11", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc85f831d-33", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:25.194 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [req-ee2e3bb9-92d3-49cc-b7f5-d4338c28b6c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-84fb0901-77a9-482c-9355-34416b3289cb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:25.378 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': '7B19C795-7A15-46CA-AAA0-4CB1473762CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15963), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:25.490 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Updated VIF entry in instance network info cache for port 5e9b878b-5ea4-4417-8d82-1b3ad6490c23. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:25.490 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Updating instance_info_cache with network_info: [{"id": "5e9b878b-5ea4-4417-8d82-1b3ad6490c23", "address": "fa:16:3e:65:9d:ec", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5e9b878b-5e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:25.509 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:25.509 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:25.509 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:25.509 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:25.533 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7B19C795-7A15-46CA-AAA0-4CB1473762CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111779), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:25.533 4076 3051114311928 MainThread DEBUG os_win._utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3557593822479248 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:25.533 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:25.561 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-d02e2221-a959-4ed8-9ad8-8ea1b83775cf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-8001e033-b435-408f-a338-cd26132a7cc1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:25.565 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-2ab27d38-7a73-4681-aac2-ae7bafca1295 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:25.565 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-2ab27d38-7a73-4681-aac2-ae7bafca1295 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:25.606 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-2ab27d38-7a73-4681-aac2-ae7bafca1295 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:25.643 4076 3051113294664 MainThread INFO nova.compute.manager [req-2ab27d38-7a73-4681-aac2-ae7bafca1295 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:25.955 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Received event network-changed-b166f44b-43d3-4457-b778-4531693659ec external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:25.959 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Refreshing instance network info cache due to event network-changed-b166f44b-43d3-4457-b778-4531693659ec. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:26.565 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': '98722FC6-5443-4082-82A2-BD06BB7EF42E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15281), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:26.647 4076 3051113294488 GreenThread-69 DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Updating instance_info_cache with network_info: [{"id": "b166f44b-43d3-4457-b778-4531693659ec", "address": "fa:16:3e:ae:31:e4", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb166f44b-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:26.682 4076 3051113294488 GreenThread-69 DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Releasing lock "refresh_cache-02a3f851-fc39-4889-aaff-e5a3385f19c0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:26.682 4076 3051113294488 GreenThread-69 DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Instance network_info: |[{"id": "b166f44b-43d3-4457-b778-4531693659ec", "address": "fa:16:3e:ae:31:e4", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb166f44b-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:26.682 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-02a3f851-fc39-4889-aaff-e5a3385f19c0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:26.682 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Refreshing network info cache for port b166f44b-43d3-4457-b778-4531693659ec _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:26.682 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Preparing to wait for external event network-vif-plugged-b166f44b-43d3-4457-b778-4531693659ec prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:26.682 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:26.682 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:26.682 4076 3051114314392 MainThread DEBUG os_win.utils.compute.vmutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Creating VM instance-00000053 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:26.694 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:26.694 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:26.694 4076 3051113294312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:26.694 4076 3051113294312 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:26.714 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '98722FC6-5443-4082-82A2-BD06BB7EF42E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:26.714 4076 3051114311928 MainThread DEBUG os_win._utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5362815856933594 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:26.730 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-be5edfe5-351a-41dc-8fd4-63061489cba9 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:26.730 4076 3051113294312 MainThread DEBUG nova.virt.hyperv.vmops [req-be5edfe5-351a-41dc-8fd4-63061489cba9 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:26.759 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-be5edfe5-351a-41dc-8fd4-63061489cba9 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:26.803 4076 3051113294312 MainThread INFO nova.compute.manager [req-be5edfe5-351a-41dc-8fd4-63061489cba9 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:27.601 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:27.759 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': 'CCA8C0BE-2E87-4CAC-8321-3B7F1761B245', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16704), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:27.880 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:27.880 4076 3051113292552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:27.905 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:27.905 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:27.905 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CCA8C0BE-2E87-4CAC-8321-3B7F1761B245', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113616), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:27.905 4076 3051114311928 MainThread DEBUG os_win._utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.726897954940796 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:27.925 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-bb5f3dd8-09f5-4b05-8b1e-a3ab70d3dc50 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:27.925 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-bb5f3dd8-09f5-4b05-8b1e-a3ab70d3dc50 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:27.973 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-bb5f3dd8-09f5-4b05-8b1e-a3ab70d3dc50 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:28.018 4076 3051113293960 MainThread INFO nova.compute.manager [req-bb5f3dd8-09f5-4b05-8b1e-a3ab70d3dc50 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:28.600 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Updated VIF entry in instance network info cache for port b166f44b-43d3-4457-b778-4531693659ec. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:28.616 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Updating instance_info_cache with network_info: [{"id": "b166f44b-43d3-4457-b778-4531693659ec", "address": "fa:16:3e:ae:31:e4", "network": {"id": "c9c0a28f-d537-4eed-ab4c-79d1152a1146", "bridge": null, "label": "tempest-MultipleCreateTestJSON-430554862-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "00a75e521b5f4d89ab74f2701a36da12", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb166f44b-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:28.616 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-22e28f61-bf9f-4c4c-b24a-92602e2299a3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-02a3f851-fc39-4889-aaff-e5a3385f19c0" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:28.928 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': 'A8987540-78F3-4B9A-B9CA-476DC8840C7C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14874), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:29.069 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:29.069 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:29.069 4076 3051113292376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:29.069 4076 3051113292376 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:29.085 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A8987540-78F3-4B9A-B9CA-476DC8840C7C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109723), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:29.085 4076 3051114311928 MainThread DEBUG os_win._utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.906946182250977 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:29.100 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-a1ae90f6-dd10-44b6-815b-35541cd1d884 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:29.100 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-a1ae90f6-dd10-44b6-815b-35541cd1d884 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:29.145 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-a1ae90f6-dd10-44b6-815b-35541cd1d884 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:29.162 4076 3051113292376 MainThread INFO nova.compute.manager [req-a1ae90f6-dd10-44b6-815b-35541cd1d884 - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:30.100 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': 'B9A03E08-B714-4FE5-A0A5-BB57BA0A68F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15112), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:30.240 4076 3051113292376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:30.240 4076 3051113292376 MainThread INFO nova.compute.manager [-] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] VM Stopped (Lifecycle Event) 2021-09-10 17:02:30.240 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:30.240 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:30.240 4076 3051114311928 MainThread ERROR os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000004f\' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000004f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)instance-0000004f16FD651E-96FB-4A3F-A97E-64110AAD3D2D%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B9A03E08-B714-4FE5-A0A5-BB57BA0A68F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114088), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\nNot enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)\n\n'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D)", 'ErrorSummaryDescription': "'instance-0000004f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000004f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:30.268 4076 3051114311928 MainThread ERROR nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Failed to change vm state of instance-0000004f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. 2021-09-10 17:02:30.268 4076 3051114311928 MainThread INFO nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Got request to destroy instance 2021-09-10 17:02:30.268 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-20f45cf1-9c0d-4a40-95dc-a7a5ab2b22aa - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:30.268 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-20f45cf1-9c0d-4a40-95dc-a7a5ab2b22aa - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:30.304 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-20f45cf1-9c0d-4a40-95dc-a7a5ab2b22aa - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:30.353 4076 3051113292376 MainThread INFO nova.compute.manager [req-20f45cf1-9c0d-4a40-95dc-a7a5ab2b22aa - - - - -] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:31.331 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed462e49-14b8-4095-9f2b-5ca95a7caa16 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Received event network-vif-plugged-c85f831d-3308-4358-afc7-f73a1171d746 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:31.331 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed462e49-14b8-4095-9f2b-5ca95a7caa16 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:31.331 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed462e49-14b8-4095-9f2b-5ca95a7caa16 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:31.331 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed462e49-14b8-4095-9f2b-5ca95a7caa16 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Processing event network-vif-plugged-c85f831d-3308-4358-afc7-f73a1171d746 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:31.427 4076 3051114314216 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Using config drive for instance 2021-09-10 17:02:31.772 4076 3051114314216 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Creating config drive at C:\openstack\instances\instance-00000052\configdrive.iso 2021-09-10 17:02:31.821 4076 3051114314216 MainThread DEBUG oslo_concurrency.processutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000052\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp9q4u6ww7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:31.955 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:32.128 4076 3051114314216 MainThread DEBUG oslo_concurrency.processutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000052\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp9q4u6ww7" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:32.317 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:02:32.317 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:32.317 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:32.346 4076 3051114311928 MainThread DEBUG os_win.utils.compute.vmutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Successfully changed vm state of instance-0000004f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:02:32.346 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Successfully changed state of VM instance-0000004f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:02:32.397 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Waiting for WMI job: {'InstanceID': 'A5CD0D9C-112E-4B73-88B0-9490586A68B3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14889), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:32.519 4076 3051114311928 MainThread DEBUG os_win.utils.jobutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] WMI job succeeded: {'InstanceID': 'A5CD0D9C-112E-4B73-88B0-9490586A68B3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29111), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:02:32.519 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:32.519 4076 3051114311928 MainThread DEBUG os_win.utils.pathutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Removing directory: C:\openstack\instances\instance-0000004f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:02:32.519 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:32.519 4076 3051114311928 MainThread ERROR nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Traceback (most recent call last): 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(*args, **kwargs) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] admin_password, network_info, block_device_info) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(self, *args, **kwds) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] should_plug_vifs=False) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'req_state': req_state}) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] time.sleep(sleep_time) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return f(*args, **kwargs) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return self._wait_for_job(job_path) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] error_desc=err_desc) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During handling of the above exception, another exception occurred: 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Traceback (most recent call last): 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] yield resources 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] accel_info=accel_info) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(*args, **kwargs) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] admin_password, network_info, block_device_info) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(self, *args, **kwds) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] should_plug_vifs=False) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'req_state': req_state}) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] time.sleep(sleep_time) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return f(*args, **kwargs) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return self._wait_for_job(job_path) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] error_desc=err_desc) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.519 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.533 4076 3051114311928 MainThread INFO nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Terminating instance 2021-09-10 17:02:32.533 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:02:32.533 4076 3051114311928 MainThread INFO nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Got request to destroy instance 2021-09-10 17:02:32.565 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Setting boot order for instance: instance-00000052: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:32.565 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.vmops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:02:32.565 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:32.565 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "instance-0000004f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:32.565 4076 3051114311928 MainThread INFO nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:02:32.565 4076 3051114311928 MainThread DEBUG nova.compute.claims [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:02:32.565 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:32.613 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:32.652 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'C04AE4FC-18BF-4D0B-9F68-8E82B7E1F275', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14909), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:32.722 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.hostops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:32.744 4076 3051114311928 MainThread DEBUG nova.virt.hyperv.hostops [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:32.789 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:32.789 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:02:32.789 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:32.789 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:32.789 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:32.789 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:32.801 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C04AE4FC-18BF-4D0B-9F68-8E82B7E1F275', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118108), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:32.801 4076 3051114314216 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18813395500183105 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:32.832 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-41964695-7567-42bd-ac97-b82170618b6e - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:32.832 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-41964695-7567-42bd-ac97-b82170618b6e - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:32.871 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-41964695-7567-42bd-ac97-b82170618b6e - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:32.924 4076 3051113293960 MainThread INFO nova.compute.manager [req-41964695-7567-42bd-ac97-b82170618b6e - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:32.924 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:32.924 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:02:32.973 4076 3051114311928 MainThread DEBUG nova.compute.provider_tree [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:32.990 4076 3051114311928 MainThread DEBUG nova.scheduler.client.report [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:32.990 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:32.990 4076 3051114311928 MainThread ERROR nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Traceback (most recent call last): 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(*args, **kwargs) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] admin_password, network_info, block_device_info) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(self, *args, **kwds) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] should_plug_vifs=False) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'req_state': req_state}) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] time.sleep(sleep_time) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return f(*args, **kwargs) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return self._wait_for_job(job_path) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] error_desc=err_desc) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] During handling of the above exception, another exception occurred: 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Traceback (most recent call last): 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] accel_info=accel_info) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(*args, **kwargs) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] admin_password, network_info, block_device_info) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return function(self, *args, **kwds) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] should_plug_vifs=False) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'req_state': req_state}) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] time.sleep(sleep_time) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self.force_reraise() 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] raise self.value 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return f(*args, **kwargs) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] return self._wait_for_job(job_path) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] error_desc=err_desc) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. 2021-09-10 17:02:32.990 4076 3051114311928 MainThread TRACE nova.compute.manager [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] 2021-09-10 17:02:32.990 4076 3051114311928 MainThread DEBUG nova.compute.utils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:02:33.005 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Build of instance 30a2e0f5-74d7-453d-ad6a-628ef91e6d39 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000004f' failed to start. Not enough memory in the system to start the virtual machine instance-0000004f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000004f' failed to start. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Not enough memory in the system to start the virtual machine instance-0000004f with ram size 1024 megabytes. (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) 'instance-0000004f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 16FD651E-96FB-4A3F-A97E-64110AAD3D2D) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:02:33.005 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:02:33.005 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:02:33.005 4076 3051114311928 MainThread DEBUG nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:02:33.005 4076 3051114311928 MainThread DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:33.846 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '72271123-A68A-497B-8F3B-BB0FA0CC0E28', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15879), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:33.978 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:33.978 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:33.978 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:33.978 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:33.993 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '72271123-A68A-497B-8F3B-BB0FA0CC0E28', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113734), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:33.993 4076 3051114314216 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3798913955688477 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:34.022 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-9cadd1cb-0c4e-4068-894c-155c11e22f7b - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:34.022 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-9cadd1cb-0c4e-4068-894c-155c11e22f7b - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:34.070 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-9cadd1cb-0c4e-4068-894c-155c11e22f7b - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:34.100 4076 3051113293960 MainThread INFO nova.compute.manager [req-9cadd1cb-0c4e-4068-894c-155c11e22f7b - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:34.506 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-f5dbf299-fb26-4928-8aa1-914bf7f12d46 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Received event network-vif-plugged-5e9b878b-5ea4-4417-8d82-1b3ad6490c23 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:34.506 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-f5dbf299-fb26-4928-8aa1-914bf7f12d46 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:34.506 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-f5dbf299-fb26-4928-8aa1-914bf7f12d46 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:34.506 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-f5dbf299-fb26-4928-8aa1-914bf7f12d46 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Processing event network-vif-plugged-5e9b878b-5ea4-4417-8d82-1b3ad6490c23 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:34.538 4076 3051114312280 MainThread INFO nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Using config drive for instance 2021-09-10 17:02:34.709 4076 3051114312280 MainThread INFO nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Creating config drive at C:\openstack\instances\instance-00000050\configdrive.iso 2021-09-10 17:02:34.768 4076 3051114312280 MainThread DEBUG oslo_concurrency.processutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000050\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4g47vftn execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:35.002 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:35.018 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:35.018 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:02:35.034 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '17E15330-5217-48C6-AF5C-5AC772E8AF16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15659), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:35.065 4076 3051114312280 MainThread DEBUG oslo_concurrency.processutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000050\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4g47vftn" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:35.090 4076 3051114311928 MainThread DEBUG nova.network.neutron [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:35.130 4076 3051114311928 MainThread INFO nova.compute.manager [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] [instance: 30a2e0f5-74d7-453d-ad6a-628ef91e6d39] Took 2.12 seconds to deallocate network for instance. 2021-09-10 17:02:35.165 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:35.165 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:35.175 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:35.175 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:35.175 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '17E15330-5217-48C6-AF5C-5AC772E8AF16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116806), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:35.175 4076 3051114314216 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.561927318572998 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:35.194 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-a032560f-a51f-4e30-a590-cd61b6ffe432 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:35.194 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-a032560f-a51f-4e30-a590-cd61b6ffe432 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:35.233 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-a032560f-a51f-4e30-a590-cd61b6ffe432 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:35.295 4076 3051113294664 MainThread INFO nova.compute.manager [req-a032560f-a51f-4e30-a590-cd61b6ffe432 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:35.474 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Setting boot order for instance: instance-00000050: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:35.504 4076 3051114311928 MainThread INFO nova.scheduler.client.report [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Deleted allocations for instance 30a2e0f5-74d7-453d-ad6a-628ef91e6d39 2021-09-10 17:02:35.539 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:35.574 4076 3051114311928 MainThread DEBUG oslo_concurrency.lockutils [req-41772b22-dcb9-4a17-abb7-f6b90cd34b21 a57565d91e984026bfef16cbf8b6f2bc 57e8809b286f4c5aad08911c6a289582 - default default] Lock "30a2e0f5-74d7-453d-ad6a-628ef91e6d39" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:35.574 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': 'C396DF50-7681-45F8-ADAC-26A7147748EC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16077), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:35.701 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:35.701 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:35.721 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:35.721 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:35.722 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C396DF50-7681-45F8-ADAC-26A7147748EC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114387), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:35.722 4076 3051114312280 MainThread DEBUG os_win._utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18277502059936523 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:35.757 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-b647d2cb-b33e-467c-b152-8aab1667125a - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:35.757 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-b647d2cb-b33e-467c-b152-8aab1667125a - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:35.806 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-b647d2cb-b33e-467c-b152-8aab1667125a - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:35.835 4076 3051113294664 MainThread INFO nova.compute.manager [req-b647d2cb-b33e-467c-b152-8aab1667125a - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:36.200 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '135AB78B-5F0A-443C-9C1B-AE56CADFDF49', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17929), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:36.237 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:36.265 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:36.334 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:36.334 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:36.334 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:36.348 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:36.348 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '135AB78B-5F0A-443C-9C1B-AE56CADFDF49', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115948), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:36.348 4076 3051114314216 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7351877689361572 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:36.378 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-46498944-8acb-4fbb-95f8-ed5d20eff14d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:36.378 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-46498944-8acb-4fbb-95f8-ed5d20eff14d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:36.441 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-46498944-8acb-4fbb-95f8-ed5d20eff14d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:36.441 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:36.441 4076 3051113294840 MainThread DEBUG nova.virt.hardware [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:36.441 4076 3051113294840 MainThread INFO nova.compute.claims [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Claim successful on node n-h1-808474-1 2021-09-10 17:02:36.486 4076 3051113294664 MainThread INFO nova.compute.manager [req-46498944-8acb-4fbb-95f8-ed5d20eff14d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:36.646 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:36.646 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:36.753 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '49427C50-2E43-4585-9897-5E4441ABFD2C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15466), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:36.783 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:36.783 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:36.866 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:36.888 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:36.888 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:36.888 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:36.888 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:36.888 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '49427C50-2E43-4585-9897-5E4441ABFD2C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118617), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:36.888 4076 3051114312280 MainThread DEBUG os_win._utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.348607063293457 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:36.940 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:36.940 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:36.940 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:36.940 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:36.940 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-c9b5f3e9-2bbb-4076-a52c-b10587af8d8f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Received event network-vif-plugged-b166f44b-43d3-4457-b778-4531693659ec external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:36.971 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-c9b5f3e9-2bbb-4076-a52c-b10587af8d8f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:36.971 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-c9b5f3e9-2bbb-4076-a52c-b10587af8d8f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:36.971 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-c9b5f3e9-2bbb-4076-a52c-b10587af8d8f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Processing event network-vif-plugged-b166f44b-43d3-4457-b778-4531693659ec _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:37.011 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:37.018 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-7133244c-c2ff-4494-90ca-9b56df8c1c64 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:37.018 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-7133244c-c2ff-4494-90ca-9b56df8c1c64 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:37.050 4076 3051114314392 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Using config drive for instance 2021-09-10 17:02:37.068 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-7133244c-c2ff-4494-90ca-9b56df8c1c64 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:37.205 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:37.205 4076 3051113294136 GreenThread-70 DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:37.205 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:37.205 4076 3051113294664 MainThread INFO nova.compute.manager [req-7133244c-c2ff-4494-90ca-9b56df8c1c64 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:37.241 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:37.381 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '0D96CDC1-ED7D-4668-9237-FC37240E16FA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15594), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:37.442 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:37.471 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Spawning new instance 2021-09-10 17:02:37.480 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:37.480 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:37.480 4076 3051113294840 MainThread DEBUG nova.block_device [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:37.480 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:37.493 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:37.493 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Creating directory: C:\openstack\instances\instance-00000054 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:37.493 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000054\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:37.504 4076 3051113292376 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:37.523 4076 3051113292376 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:37.523 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:37.523 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:37.523 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0D96CDC1-ED7D-4668-9237-FC37240E16FA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113082), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:37.523 4076 3051114314216 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9095470905303955 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:37.523 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Resizing VHD C:\openstack\instances\instance-00000054\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:37.523 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-53c0d117-198f-4658-915c-f6904f206135 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:37.523 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-53c0d117-198f-4658-915c-f6904f206135 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:37.592 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-53c0d117-198f-4658-915c-f6904f206135 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:37.627 4076 3051113292376 MainThread INFO nova.compute.manager [req-53c0d117-198f-4658-915c-f6904f206135 - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:37.830 4076 3051114314392 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Creating config drive at C:\openstack\instances\instance-00000053\configdrive.iso 2021-09-10 17:02:37.865 4076 3051114314392 MainThread DEBUG oslo_concurrency.processutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000053\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp52hqmq3i execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:38.012 4076 3051113294136 GreenThread-70 DEBUG nova.policy [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '206940e6bc1a478bb7f92f7778bec0e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5283391577e14a77bbd03e5dcf9f9e5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:38.033 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': 'B777FE17-C89C-4834-8288-DB8DB0DC3C61', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15256), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:38.181 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:38.181 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:38.181 4076 3051113295192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:38.181 4076 3051113295192 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:38.181 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B777FE17-C89C-4834-8288-DB8DB0DC3C61', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113281), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:38.181 4076 3051114312280 MainThread DEBUG os_win._utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6417500972747803 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:38.181 4076 3051114314392 MainThread DEBUG oslo_concurrency.processutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000053\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp52hqmq3i" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:38.192 4076 3051113295192 MainThread DEBUG nova.compute.manager [req-0cdf0757-f9ae-4a8a-bda4-c5e614554008 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:38.192 4076 3051113295192 MainThread DEBUG nova.virt.hyperv.vmops [req-0cdf0757-f9ae-4a8a-bda4-c5e614554008 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:38.256 4076 3051113295192 MainThread DEBUG nova.compute.manager [req-0cdf0757-f9ae-4a8a-bda4-c5e614554008 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:38.302 4076 3051113295192 MainThread INFO nova.compute.manager [req-0cdf0757-f9ae-4a8a-bda4-c5e614554008 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:38.549 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'FFE36CA5-1419-46DA-9D5C-4FB163101D1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14959), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:38.583 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Setting boot order for instance: instance-00000053: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:38.649 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:38.678 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '1A14117E-CC92-4748-A633-AC2EE209C09E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15114), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:38.678 4076 3051113295192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:38.678 4076 3051113295192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:38.678 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:38.678 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] VM Stopped (Lifecycle Event) 2021-09-10 17:02:38.695 4076 3051114314216 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000052\' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000052\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)instance-00000052C5B7BA24-4290-4358-8D14-39C4937B0EB9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FFE36CA5-1419-46DA-9D5C-4FB163101D1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111951), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\nNot enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)\n\n'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9)", 'ErrorSummaryDescription': "'instance-00000052' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000052.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:38.695 4076 3051114314216 MainThread ERROR nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Failed to change vm state of instance-00000052 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. 2021-09-10 17:02:38.695 4076 3051114314216 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Got request to destroy instance 2021-09-10 17:02:38.709 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-ec98e250-ad5c-4b11-997d-764eab57e21d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:38.709 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-ec98e250-ad5c-4b11-997d-764eab57e21d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:38.756 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-ec98e250-ad5c-4b11-997d-764eab57e21d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:38.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:38.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:38.787 4076 3051113292728 MainThread INFO nova.compute.manager [req-ec98e250-ad5c-4b11-997d-764eab57e21d - - - - -] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:38.815 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:38.815 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:38.815 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:02:38.815 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:38.815 4076 3051113295192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:38.815 4076 3051113295192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:38.815 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:38.815 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:38.831 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1A14117E-CC92-4748-A633-AC2EE209C09E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116865), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:38.831 4076 3051114314392 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18190646171569824 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:38.831 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:38.831 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-983c0481-8139-4efd-9775-93b6d8db0c81 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:38.831 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-983c0481-8139-4efd-9775-93b6d8db0c81 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:38.906 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-983c0481-8139-4efd-9775-93b6d8db0c81 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:38.931 4076 3051113292728 MainThread INFO nova.compute.manager [req-983c0481-8139-4efd-9775-93b6d8db0c81 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:39.081 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2185MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:02:39.081 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:39.211 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '68568233-DBC1-466E-809A-07027F8F4DBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14991), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:39.336 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:39.336 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:39.336 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:39.336 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 8001e033-b435-408f-a338-cd26132a7cc1 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 84fb0901-77a9-482c-9355-34416b3289cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 02a3f851-fc39-4889-aaff-e5a3385f19c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance d358307f-112b-4be6-9b2a-08f9ad26205e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 16 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:02:39.362 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=16896MB phys_disk=79GB used_disk=16GB total_vcpus=6 used_vcpus=16 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:02:39.381 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '68568233-DBC1-466E-809A-07027F8F4DBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115016), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:39.381 4076 3051114312280 MainThread DEBUG os_win._utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8415887355804443 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:39.381 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-0bb1d5ce-f4c1-4eae-a7b3-6b517b3a0716 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:39.381 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-0bb1d5ce-f4c1-4eae-a7b3-6b517b3a0716 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:39.404 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:39.415 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:39.433 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-0bb1d5ce-f4c1-4eae-a7b3-6b517b3a0716 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:39.478 4076 3051113293960 MainThread INFO nova.compute.manager [req-0bb1d5ce-f4c1-4eae-a7b3-6b517b3a0716 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:39.630 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:39.677 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:39.677 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:02:39.677 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:39.736 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:39.783 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:39.877 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '12ADCC9F-8FA0-47ED-9391-FA08DC0E6511', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19220), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:39.894 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:39.894 4076 3051113292728 MainThread DEBUG nova.virt.hardware [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:39.894 4076 3051113292728 MainThread INFO nova.compute.claims [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Claim successful on node n-h1-808474-1 2021-09-10 17:02:40.018 4076 3051113293960 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:40.018 4076 3051113293960 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:40.018 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.018 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.018 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '12ADCC9F-8FA0-47ED-9391-FA08DC0E6511', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114395), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:40.018 4076 3051114314392 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.368488073348999 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:40.037 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-3d36063e-8480-4ceb-9960-6ea0156217fe - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:40.037 4076 3051113293960 MainThread DEBUG nova.virt.hyperv.vmops [req-3d36063e-8480-4ceb-9960-6ea0156217fe - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:40.085 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-3d36063e-8480-4ceb-9960-6ea0156217fe - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:40.085 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.hostops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:40.113 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.hostops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:40.113 4076 3051113293960 MainThread INFO nova.compute.manager [req-3d36063e-8480-4ceb-9960-6ea0156217fe - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:40.317 4076 3051113292728 MainThread DEBUG nova.compute.provider_tree [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:40.346 4076 3051113292728 MainThread DEBUG nova.scheduler.client.report [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:40.346 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.346 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:40.402 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': 'D38B3DC8-FCB3-4FA1-BD11-7D0B59DDC11F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15066), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:40.457 4076 3051113292728 MainThread DEBUG nova.compute.utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:40.457 4076 3051113293960 GreenThread-71 DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:40.457 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:40.494 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:40.552 4076 3051113295368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:40.552 4076 3051113295368 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:40.552 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.552 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.552 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D38B3DC8-FCB3-4FA1-BD11-7D0B59DDC11F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124075), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:40.552 4076 3051114312280 MainThread DEBUG os_win._utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.012919664382935 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:40.569 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-5a0c05e7-9949-4a7a-8e83-a7d768a48cc5 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:40.569 4076 3051113295368 MainThread DEBUG nova.virt.hyperv.vmops [req-5a0c05e7-9949-4a7a-8e83-a7d768a48cc5 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:40.614 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-5a0c05e7-9949-4a7a-8e83-a7d768a48cc5 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:40.658 4076 3051113295368 MainThread INFO nova.compute.manager [req-5a0c05e7-9949-4a7a-8e83-a7d768a48cc5 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:40.721 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:02:40.721 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.721 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.721 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:40.721 4076 3051113292728 MainThread INFO nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Spawning new instance 2021-09-10 17:02:40.741 4076 3051114314216 MainThread DEBUG os_win.utils.compute.vmutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed vm state of instance-00000052 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:02:40.741 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed state of VM instance-00000052 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG nova.block_device [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG os_win.utils.pathutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-00000055 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:40.741 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000055\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:40.790 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Resizing VHD C:\openstack\instances\instance-00000055\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:40.790 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '01D794E2-941C-4C15-9366-1AF70282EE76', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16360), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:40.851 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Successfully created port: bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:40.893 4076 3051114314216 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job succeeded: {'InstanceID': '01D794E2-941C-4C15-9366-1AF70282EE76', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31965), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:02:40.893 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.893 4076 3051114314216 MainThread DEBUG os_win.utils.pathutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Removing directory: C:\openstack\instances\instance-00000052 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:02:40.893 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.893 4076 3051114314216 MainThread ERROR nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Traceback (most recent call last): 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(*args, **kwargs) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] admin_password, network_info, block_device_info) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(self, *args, **kwds) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] should_plug_vifs=False) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'req_state': req_state}) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] time.sleep(sleep_time) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return f(*args, **kwargs) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return self._wait_for_job(job_path) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] error_desc=err_desc) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During handling of the above exception, another exception occurred: 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Traceback (most recent call last): 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] yield resources 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] accel_info=accel_info) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(*args, **kwargs) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] admin_password, network_info, block_device_info) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(self, *args, **kwds) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] should_plug_vifs=False) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'req_state': req_state}) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] time.sleep(sleep_time) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return f(*args, **kwargs) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return self._wait_for_job(job_path) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] error_desc=err_desc) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:40.893 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:40.909 4076 3051114314216 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Terminating instance 2021-09-10 17:02:40.909 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:02:40.909 4076 3051114314216 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Got request to destroy instance 2021-09-10 17:02:40.928 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:02:40.928 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:40.928 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000052" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:40.928 4076 3051114314216 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:02:40.928 4076 3051114314216 MainThread DEBUG nova.compute.claims [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:02:40.928 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:41.053 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:41.053 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '7A1CFF32-EE6E-457C-A329-49635DDBA1B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15257), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:41.053 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:41.146 4076 3051113293960 GreenThread-71 DEBUG nova.policy [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:41.177 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:41.177 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:41.177 4076 3051113295368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:41.177 4076 3051113295368 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:41.195 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7A1CFF32-EE6E-457C-A329-49635DDBA1B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114047), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:41.195 4076 3051114314392 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.545964002609253 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:41.221 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-ec9b275e-5211-4b5b-aa32-71cbfee5f5ff - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:41.221 4076 3051113295368 MainThread DEBUG nova.virt.hyperv.vmops [req-ec9b275e-5211-4b5b-aa32-71cbfee5f5ff - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:41.261 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-ec9b275e-5211-4b5b-aa32-71cbfee5f5ff - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:41.283 4076 3051114314216 MainThread DEBUG nova.compute.provider_tree [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:41.305 4076 3051114314216 MainThread ERROR nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Traceback (most recent call last): 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(*args, **kwargs) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] admin_password, network_info, block_device_info) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(self, *args, **kwds) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] should_plug_vifs=False) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'req_state': req_state}) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] time.sleep(sleep_time) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return f(*args, **kwargs) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return self._wait_for_job(job_path) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] error_desc=err_desc) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] During handling of the above exception, another exception occurred: 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Traceback (most recent call last): 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] accel_info=accel_info) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(*args, **kwargs) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] admin_password, network_info, block_device_info) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return function(self, *args, **kwds) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] should_plug_vifs=False) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'req_state': req_state}) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] time.sleep(sleep_time) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self.force_reraise() 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] raise self.value 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return f(*args, **kwargs) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] return self._wait_for_job(job_path) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] error_desc=err_desc) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. 2021-09-10 17:02:41.305 4076 3051114314216 MainThread TRACE nova.compute.manager [instance: 84fb0901-77a9-482c-9355-34416b3289cb] 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.compute.utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Build of instance 84fb0901-77a9-482c-9355-34416b3289cb was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000052' failed to start. Not enough memory in the system to start the virtual machine instance-00000052. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000052' failed to start. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Not enough memory in the system to start the virtual machine instance-00000052 with ram size 1024 megabytes. (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) 'instance-00000052' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C5B7BA24-4290-4358-8D14-39C4937B0EB9) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:02:41.305 4076 3051114314216 MainThread DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:41.305 4076 3051113295368 MainThread INFO nova.compute.manager [req-ec9b275e-5211-4b5b-aa32-71cbfee5f5ff - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:41.583 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': 'C48CC24A-FBE3-4761-8584-BBADC8FE33E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15852), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:41.687 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:41.687 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:02:41.710 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:41.732 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] VM Stopped (Lifecycle Event) 2021-09-10 17:02:41.732 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:41.732 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:41.732 4076 3051114312280 MainThread ERROR os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000050\' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000050\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)instance-00000050C7CCC588-518C-4716-99B4-EB328050E6EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C48CC24A-FBE3-4761-8584-BBADC8FE33E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111318), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\nNot enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)\n\n'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF)", 'ErrorSummaryDescription': "'instance-00000050' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000050.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:41.732 4076 3051114312280 MainThread ERROR nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Failed to change vm state of instance-00000050 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. 2021-09-10 17:02:41.732 4076 3051114312280 MainThread INFO nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Got request to destroy instance 2021-09-10 17:02:41.792 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-47bc582e-963e-49e8-a81a-157597e85d66 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:41.792 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-47bc582e-963e-49e8-a81a-157597e85d66 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:41.792 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:02:41.836 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-47bc582e-963e-49e8-a81a-157597e85d66 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:41.866 4076 3051055215720 MainThread INFO nova.compute.manager [req-47bc582e-963e-49e8-a81a-157597e85d66 - - - - -] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:42.007 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:42.034 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:42.154 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:42.158 4076 3051093170312 MainThread DEBUG nova.virt.hardware [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:42.158 4076 3051093170312 MainThread INFO nova.compute.claims [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Claim successful on node n-h1-808474-1 2021-09-10 17:02:42.236 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '98509D83-A7A4-4826-A626-7CB355F7FE1E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14895), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:42.348 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:42.367 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:42.367 4076 3051093170840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:42.367 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:42.367 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:42.367 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:42.387 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '98509D83-A7A4-4826-A626-7CB355F7FE1E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117813), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:42.387 4076 3051114314392 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.737492322921753 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:42.413 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-20082fe0-3ee4-49e3-a7e9-ec348f5c9816 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:42.413 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-20082fe0-3ee4-49e3-a7e9-ec348f5c9816 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:42.440 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-20082fe0-3ee4-49e3-a7e9-ec348f5c9816 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:42.488 4076 3051055214664 MainThread INFO nova.compute.manager [req-20082fe0-3ee4-49e3-a7e9-ec348f5c9816 - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:42.589 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:42.615 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:42.615 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.454s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:42.615 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:42.723 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:42.739 4076 3051055214664 GreenThread-72 DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:42.739 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:42.970 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:42.986 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Spawning new instance 2021-09-10 17:02:42.986 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG nova.block_device [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Creating directory: C:\openstack\instances\instance-00000051 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:43.007 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000051\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:43.044 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Resizing VHD C:\openstack\instances\instance-00000051\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:43.138 4076 3051055214664 GreenThread-72 DEBUG nova.policy [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0063b05acec342499daf8129dc0ca365', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d6026a33839407f9f68cb295fbb6928', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:43.154 4076 3051114314216 MainThread DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:43.205 4076 3051114314216 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 84fb0901-77a9-482c-9355-34416b3289cb] Took 1.91 seconds to deallocate network for instance. 2021-09-10 17:02:43.426 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'E0729A09-BFFB-4C2A-9954-13BED3FC0D25', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14836), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:43.444 4076 3051114314216 MainThread INFO nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Deleted allocations for instance 84fb0901-77a9-482c-9355-34416b3289cb 2021-09-10 17:02:43.494 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "84fb0901-77a9-482c-9355-34416b3289cb" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.797s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.570 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.570 4076 3051093169784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.570 4076 3051093170840 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:43.570 4076 3051093170840 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:43.570 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E0729A09-BFFB-4C2A-9954-13BED3FC0D25', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111521), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:43.570 4076 3051114314392 MainThread DEBUG os_win._utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.92065691947937 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:43.600 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-aaf0eded-76bb-4210-9eb2-384e7345f2cd - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:43.600 4076 3051093170840 MainThread DEBUG nova.virt.hyperv.vmops [req-aaf0eded-76bb-4210-9eb2-384e7345f2cd - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:43.649 4076 3051093170840 MainThread DEBUG nova.compute.manager [req-aaf0eded-76bb-4210-9eb2-384e7345f2cd - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:43.663 4076 3051093170840 MainThread INFO nova.compute.manager [req-aaf0eded-76bb-4210-9eb2-384e7345f2cd - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:43.663 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:02:43.663 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.663 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.686 4076 3051114312280 MainThread DEBUG os_win.utils.compute.vmutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Successfully changed vm state of instance-00000050 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:02:43.686 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Successfully changed state of VM instance-00000050 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:02:43.741 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '4FE0A57A-BD86-4FA2-A6A1-AD80DAE531A0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14467), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:43.850 4076 3051114312280 MainThread DEBUG os_win.utils.jobutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job succeeded: {'InstanceID': '4FE0A57A-BD86-4FA2-A6A1-AD80DAE531A0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30412), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:02:43.850 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.850 4076 3051114312280 MainThread DEBUG os_win.utils.pathutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Removing directory: C:\openstack\instances\instance-00000050 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:02:43.850 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.850 4076 3051114312280 MainThread ERROR nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Traceback (most recent call last): 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(*args, **kwargs) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] admin_password, network_info, block_device_info) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(self, *args, **kwds) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] should_plug_vifs=False) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'req_state': req_state}) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] time.sleep(sleep_time) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return f(*args, **kwargs) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return self._wait_for_job(job_path) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] error_desc=err_desc) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During handling of the above exception, another exception occurred: 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Traceback (most recent call last): 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] yield resources 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] accel_info=accel_info) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(*args, **kwargs) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] admin_password, network_info, block_device_info) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(self, *args, **kwds) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] should_plug_vifs=False) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'req_state': req_state}) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] time.sleep(sleep_time) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return f(*args, **kwargs) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return self._wait_for_job(job_path) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] error_desc=err_desc) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:43.850 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:43.865 4076 3051114312280 MainThread INFO nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Terminating instance 2021-09-10 17:02:43.865 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:02:43.865 4076 3051114312280 MainThread INFO nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Got request to destroy instance 2021-09-10 17:02:43.865 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:02:43.887 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:43.887 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000050" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:43.887 4076 3051114312280 MainThread INFO nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:02:43.887 4076 3051114312280 MainThread DEBUG nova.compute.claims [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:02:43.887 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:44.065 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.hostops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:44.081 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.hostops [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:44.283 4076 3051114312280 MainThread DEBUG nova.compute.provider_tree [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.scheduler.client.report [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:44.309 4076 3051114312280 MainThread ERROR nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Traceback (most recent call last): 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(*args, **kwargs) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] admin_password, network_info, block_device_info) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(self, *args, **kwds) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] should_plug_vifs=False) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'req_state': req_state}) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] time.sleep(sleep_time) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return f(*args, **kwargs) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return self._wait_for_job(job_path) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] error_desc=err_desc) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] During handling of the above exception, another exception occurred: 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Traceback (most recent call last): 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] accel_info=accel_info) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(*args, **kwargs) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] admin_password, network_info, block_device_info) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return function(self, *args, **kwds) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] should_plug_vifs=False) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'req_state': req_state}) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] time.sleep(sleep_time) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self.force_reraise() 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] raise self.value 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return f(*args, **kwargs) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] return self._wait_for_job(job_path) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] error_desc=err_desc) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. 2021-09-10 17:02:44.309 4076 3051114312280 MainThread TRACE nova.compute.manager [instance: 8001e033-b435-408f-a338-cd26132a7cc1] 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.compute.utils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Build of instance 8001e033-b435-408f-a338-cd26132a7cc1 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000050' failed to start. Not enough memory in the system to start the virtual machine instance-00000050. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000050' failed to start. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Not enough memory in the system to start the virtual machine instance-00000050 with ram size 1024 megabytes. (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) 'instance-00000050' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID C7CCC588-518C-4716-99B4-EB328050E6EF) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:02:44.309 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:44.584 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': 'D214DA1C-E948-4479-AD65-3916841FF8C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15522), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:44.725 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:44.725 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:44.725 4076 3051093171192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:44.725 4076 3051093171192 MainThread INFO nova.compute.manager [-] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] VM Stopped (Lifecycle Event) 2021-09-10 17:02:44.725 4076 3051114314392 MainThread ERROR os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000053\' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000053\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)instance-00000053A2AEC973-19D6-44F7-824A-DBD6AD57D8E6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D214DA1C-E948-4479-AD65-3916841FF8C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112726), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\nNot enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)\n\n'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6)", 'ErrorSummaryDescription': "'instance-00000053' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000053.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:44.725 4076 3051114314392 MainThread ERROR nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Failed to change vm state of instance-00000053 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. 2021-09-10 17:02:44.725 4076 3051114314392 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Got request to destroy instance 2021-09-10 17:02:44.767 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-46b8d37b-692d-4ab1-a19f-a06de3ab8a0b - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:44.767 4076 3051093171192 MainThread DEBUG nova.virt.hyperv.vmops [req-46b8d37b-692d-4ab1-a19f-a06de3ab8a0b - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:44.819 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-46b8d37b-692d-4ab1-a19f-a06de3ab8a0b - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:44.850 4076 3051093171192 MainThread INFO nova.compute.manager [req-46b8d37b-692d-4ab1-a19f-a06de3ab8a0b - - - - -] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:44.912 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Successfully updated port: bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:44.928 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Successfully created port: 501733fd-1af5-48d6-aec1-204bd50f155d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:44.943 4076 3051113294136 GreenThread-70 DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Acquired lock "refresh_cache-d358307f-112b-4be6-9b2a-08f9ad26205e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:44.956 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:45.349 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:45.745 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Successfully created port: d758a788-2d3f-46ad-b1bd-0ee3066bfc4e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:02:46.567 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:46.600 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:02:46.600 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:46.600 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:46.600 4076 3051114312280 MainThread INFO nova.compute.manager [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: 8001e033-b435-408f-a338-cd26132a7cc1] Took 2.28 seconds to deallocate network for instance. 2021-09-10 17:02:46.613 4076 3051114314392 MainThread DEBUG os_win.utils.compute.vmutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed vm state of instance-00000053 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:02:46.613 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Successfully changed state of VM instance-00000053 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:02:46.668 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Waiting for WMI job: {'InstanceID': '8B78C932-5650-4F1D-BC8D-C2926C589C55', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14561), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:46.802 4076 3051114314392 MainThread DEBUG os_win.utils.jobutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] WMI job succeeded: {'InstanceID': '8B78C932-5650-4F1D-BC8D-C2926C589C55', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28205), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:02:46.802 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:46.802 4076 3051114314392 MainThread DEBUG os_win.utils.pathutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Removing directory: C:\openstack\instances\instance-00000053 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:02:46.802 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:46.802 4076 3051114314392 MainThread ERROR nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Traceback (most recent call last): 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(*args, **kwargs) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] admin_password, network_info, block_device_info) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(self, *args, **kwds) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] should_plug_vifs=False) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'req_state': req_state}) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] time.sleep(sleep_time) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return f(*args, **kwargs) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return self._wait_for_job(job_path) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] error_desc=err_desc) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During handling of the above exception, another exception occurred: 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Traceback (most recent call last): 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] yield resources 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] accel_info=accel_info) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(*args, **kwargs) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] admin_password, network_info, block_device_info) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(self, *args, **kwds) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] should_plug_vifs=False) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'req_state': req_state}) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] time.sleep(sleep_time) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return f(*args, **kwargs) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return self._wait_for_job(job_path) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] error_desc=err_desc) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:46.802 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:46.821 4076 3051114314392 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Terminating instance 2021-09-10 17:02:46.821 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:02:46.821 4076 3051114314392 MainThread INFO nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Got request to destroy instance 2021-09-10 17:02:46.841 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.vmops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:02:46.841 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:46.841 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "instance-00000053" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:46.841 4076 3051114314392 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:02:46.841 4076 3051114314392 MainThread DEBUG nova.compute.claims [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:02:46.841 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:46.959 4076 3051114312280 MainThread INFO nova.scheduler.client.report [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Deleted allocations for instance 8001e033-b435-408f-a338-cd26132a7cc1 2021-09-10 17:02:47.017 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-3b5a268a-8c60-404d-8378-b380f189b636 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "8001e033-b435-408f-a338-cd26132a7cc1" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 34.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:47.053 4076 3051113294136 GreenThread-70 DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Updating instance_info_cache with network_info: [{"id": "bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f", "address": "fa:16:3e:9f:53:5e", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb7fe2b1-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:47.097 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:47.097 4076 3051114314392 MainThread DEBUG nova.virt.hyperv.hostops [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:47.115 4076 3051113294136 GreenThread-70 DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Releasing lock "refresh_cache-d358307f-112b-4be6-9b2a-08f9ad26205e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:47.115 4076 3051113294136 GreenThread-70 DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Instance network_info: |[{"id": "bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f", "address": "fa:16:3e:9f:53:5e", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb7fe2b1-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:47.115 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Preparing to wait for external event network-vif-plugged-bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:47.115 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:47.115 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:47.115 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Creating VM instance-00000054 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:47.319 4076 3051114314392 MainThread DEBUG nova.compute.provider_tree [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:47.343 4076 3051114314392 MainThread DEBUG nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:47.343 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:47.343 4076 3051114314392 MainThread ERROR nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Traceback (most recent call last): 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(*args, **kwargs) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] admin_password, network_info, block_device_info) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(self, *args, **kwds) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] should_plug_vifs=False) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'req_state': req_state}) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] time.sleep(sleep_time) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return f(*args, **kwargs) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return self._wait_for_job(job_path) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] error_desc=err_desc) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] During handling of the above exception, another exception occurred: 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Traceback (most recent call last): 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] accel_info=accel_info) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(*args, **kwargs) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] admin_password, network_info, block_device_info) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return function(self, *args, **kwds) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] should_plug_vifs=False) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'req_state': req_state}) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] time.sleep(sleep_time) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self.force_reraise() 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] raise self.value 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return f(*args, **kwargs) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] return self._wait_for_job(job_path) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] error_desc=err_desc) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. 2021-09-10 17:02:47.343 4076 3051114314392 MainThread TRACE nova.compute.manager [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.compute.utils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Build of instance 02a3f851-fc39-4889-aaff-e5a3385f19c0 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000053' failed to start. Not enough memory in the system to start the virtual machine instance-00000053. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000053' failed to start. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Not enough memory in the system to start the virtual machine instance-00000053 with ram size 1024 megabytes. (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) 'instance-00000053' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A2AEC973-19D6-44F7-824A-DBD6AD57D8E6) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:02:47.353 4076 3051114314392 MainThread DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:02:47.677 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Successfully updated port: d758a788-2d3f-46ad-b1bd-0ee3066bfc4e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:47.709 4076 3051055214664 GreenThread-72 DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Acquired lock "refresh_cache-c4dcbaf6-eadf-45db-9755-790791af2bf7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:47.709 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:47.798 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Successfully updated port: 501733fd-1af5-48d6-aec1-204bd50f155d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:02:47.830 4076 3051113293960 GreenThread-71 DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-ba57675b-1666-430f-bbaa-84c5ca720900" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:47.830 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:02:47.830 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:48.005 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:48.148 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:02:48.767 4076 3051093169960 MainThread DEBUG nova.compute.manager [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Received event network-changed-bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:48.767 4076 3051093169960 MainThread DEBUG nova.compute.manager [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Refreshing instance network info cache due to event network-changed-bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:48.767 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d358307f-112b-4be6-9b2a-08f9ad26205e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:48.767 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Refreshing network info cache for port bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:48.950 4076 3051055214664 GreenThread-72 DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Updating instance_info_cache with network_info: [{"id": "d758a788-2d3f-46ad-b1bd-0ee3066bfc4e", "address": "fa:16:3e:50:ca:2c", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd758a788-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:48.973 4076 3051055214664 GreenThread-72 DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Releasing lock "refresh_cache-c4dcbaf6-eadf-45db-9755-790791af2bf7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:48.973 4076 3051055214664 GreenThread-72 DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Instance network_info: |[{"id": "d758a788-2d3f-46ad-b1bd-0ee3066bfc4e", "address": "fa:16:3e:50:ca:2c", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd758a788-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:48.973 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Preparing to wait for external event network-vif-plugged-d758a788-2d3f-46ad-b1bd-0ee3066bfc4e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:48.973 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:48.973 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:48.973 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Creating VM instance-00000051 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:49.115 4076 3051113293960 GreenThread-71 DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Updating instance_info_cache with network_info: [{"id": "501733fd-1af5-48d6-aec1-204bd50f155d", "address": "fa:16:3e:4f:e0:a3", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap501733fd-1a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:49.147 4076 3051113293960 GreenThread-71 DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-ba57675b-1666-430f-bbaa-84c5ca720900" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:49.147 4076 3051113293960 GreenThread-71 DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Instance network_info: |[{"id": "501733fd-1af5-48d6-aec1-204bd50f155d", "address": "fa:16:3e:4f:e0:a3", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap501733fd-1a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:02:49.147 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Preparing to wait for external event network-vif-plugged-501733fd-1af5-48d6-aec1-204bd50f155d prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:02:49.147 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:49.147 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:49.147 4076 3051113292728 MainThread DEBUG os_win.utils.compute.vmutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-00000055 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:02:49.357 4076 3051114314392 MainThread DEBUG nova.network.neutron [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:49.381 4076 3051114314392 MainThread INFO nova.compute.manager [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] [instance: 02a3f851-fc39-4889-aaff-e5a3385f19c0] Took 2.03 seconds to deallocate network for instance. 2021-09-10 17:02:49.670 4076 3051114314392 MainThread INFO nova.scheduler.client.report [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Deleted allocations for instance 02a3f851-fc39-4889-aaff-e5a3385f19c0 2021-09-10 17:02:49.707 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-2d5a0231-1a13-447a-9dcf-69eac5e08631 b207fe385eb647a4acf0331024dcf245 00a75e521b5f4d89ab74f2701a36da12 - default default] Lock "02a3f851-fc39-4889-aaff-e5a3385f19c0" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 33.922s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:49.883 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:50.100 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:02:50.309 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Updated VIF entry in instance network info cache for port bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:50.309 4076 3051093169960 MainThread DEBUG nova.network.neutron [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Updating instance_info_cache with network_info: [{"id": "bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f", "address": "fa:16:3e:9f:53:5e", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb7fe2b1-ed", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:50.343 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [req-b83df16d-a3c1-47c2-8164-f28e4deac9fe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d358307f-112b-4be6-9b2a-08f9ad26205e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:51.298 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Received event network-changed-d758a788-2d3f-46ad-b1bd-0ee3066bfc4e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:51.298 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Refreshing instance network info cache due to event network-changed-d758a788-2d3f-46ad-b1bd-0ee3066bfc4e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:51.298 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-c4dcbaf6-eadf-45db-9755-790791af2bf7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:51.298 4076 3051093171544 MainThread DEBUG nova.network.neutron [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Refreshing network info cache for port d758a788-2d3f-46ad-b1bd-0ee3066bfc4e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:51.393 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Received event network-changed-501733fd-1af5-48d6-aec1-204bd50f155d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:51.393 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Refreshing instance network info cache due to event network-changed-501733fd-1af5-48d6-aec1-204bd50f155d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:02:51.393 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ba57675b-1666-430f-bbaa-84c5ca720900" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:02:51.393 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Refreshing network info cache for port 501733fd-1af5-48d6-aec1-204bd50f155d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:02:52.771 4076 3051093171544 MainThread DEBUG nova.network.neutron [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Updated VIF entry in instance network info cache for port d758a788-2d3f-46ad-b1bd-0ee3066bfc4e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:52.771 4076 3051093171544 MainThread DEBUG nova.network.neutron [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Updating instance_info_cache with network_info: [{"id": "d758a788-2d3f-46ad-b1bd-0ee3066bfc4e", "address": "fa:16:3e:50:ca:2c", "network": {"id": "7381cc3b-ab27-4454-8a5b-95f2bef2e954", "bridge": null, "label": "tempest-ServerRescueNegativeTestJSON-1471687160-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5d6026a33839407f9f68cb295fbb6928", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd758a788-2d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:52.817 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [req-b3ebc804-3d36-46c4-a06f-390d19888b4e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-c4dcbaf6-eadf-45db-9755-790791af2bf7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:54.172 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Updated VIF entry in instance network info cache for port 501733fd-1af5-48d6-aec1-204bd50f155d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:02:54.172 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Updating instance_info_cache with network_info: [{"id": "501733fd-1af5-48d6-aec1-204bd50f155d", "address": "fa:16:3e:4f:e0:a3", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap501733fd-1a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:02:54.222 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-0e54ab9c-1b14-4f0d-b624-4278dd8f5582 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ba57675b-1666-430f-bbaa-84c5ca720900" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:02:54.240 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5fc321ad-7c13-46e0-aa14-32d1dcd4fa5f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Received event network-vif-plugged-bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:54.240 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5fc321ad-7c13-46e0-aa14-32d1dcd4fa5f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:54.240 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5fc321ad-7c13-46e0-aa14-32d1dcd4fa5f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:54.240 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5fc321ad-7c13-46e0-aa14-32d1dcd4fa5f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Processing event network-vif-plugged-bb7fe2b1-ed52-4683-bdbf-ba4b08882a2f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:54.284 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Using config drive for instance 2021-09-10 17:02:54.724 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Creating config drive at C:\openstack\instances\instance-00000054\configdrive.iso 2021-09-10 17:02:54.770 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000054\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkmxtccq3 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:55.127 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000054\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkmxtccq3" returned: 0 in 0.359s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:02:55.537 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Setting boot order for instance: instance-00000054: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:02:55.572 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:02:55.612 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': 'A2AFDFC9-6C49-4457-84D1-4F07182B4DD5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14952), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:55.752 4076 3051093171544 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:55.752 4076 3051093171544 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:02:55.752 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:55.752 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:55.752 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A2AFDFC9-6C49-4457-84D1-4F07182B4DD5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112840), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:55.752 4076 3051113294840 MainThread DEBUG os_win._utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17945456504821777 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:55.783 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-1833a781-51e9-4bba-8f48-a675bb64a752 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:55.783 4076 3051093171544 MainThread DEBUG nova.virt.hyperv.vmops [req-1833a781-51e9-4bba-8f48-a675bb64a752 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:55.833 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-1833a781-51e9-4bba-8f48-a675bb64a752 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:55.833 4076 3051093171544 MainThread INFO nova.compute.manager [req-1833a781-51e9-4bba-8f48-a675bb64a752 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:56.771 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '338E7F85-3434-4973-8A43-772DA4DE0AC2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15127), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:56.915 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:56.915 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:56.915 4076 3051093171544 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:56.915 4076 3051093171544 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:02:56.915 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '338E7F85-3434-4973-8A43-772DA4DE0AC2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110485), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:56.915 4076 3051113294840 MainThread DEBUG os_win._utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3421432971954346 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:56.939 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-2fdf161b-9a00-45bd-8117-8119c1a46225 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:56.939 4076 3051093171544 MainThread DEBUG nova.virt.hyperv.vmops [req-2fdf161b-9a00-45bd-8117-8119c1a46225 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:56.980 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-2fdf161b-9a00-45bd-8117-8119c1a46225 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:57.021 4076 3051093171544 MainThread INFO nova.compute.manager [req-2fdf161b-9a00-45bd-8117-8119c1a46225 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:57.934 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': 'F684AB65-8A62-44D3-A325-D3E316323135', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15063), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:58.069 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:58.069 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:58.069 4076 3051093171544 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:58.069 4076 3051093171544 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:02:58.081 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F684AB65-8A62-44D3-A325-D3E316323135', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111648), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:58.081 4076 3051113294840 MainThread DEBUG os_win._utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5087363719940186 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:58.081 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-fc1cb8a0-fa02-4a3b-bd7f-a976ac1cb24f - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:58.081 4076 3051093171544 MainThread DEBUG nova.virt.hyperv.vmops [req-fc1cb8a0-fa02-4a3b-bd7f-a976ac1cb24f - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:58.134 4076 3051093171544 MainThread DEBUG nova.compute.manager [req-fc1cb8a0-fa02-4a3b-bd7f-a976ac1cb24f - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:58.170 4076 3051093171544 MainThread INFO nova.compute.manager [req-fc1cb8a0-fa02-4a3b-bd7f-a976ac1cb24f - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:58.667 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:58.705 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:02:58.799 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:58.799 4076 3051093168552 MainThread DEBUG nova.virt.hardware [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:02:58.799 4076 3051093168552 MainThread INFO nova.compute.claims [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Claim successful on node n-h1-808474-1 2021-09-10 17:02:58.924 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:02:58.924 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:02:59.099 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '6A12BD97-581E-4353-8C14-EB27DDF7A97B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14837), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:02:59.146 4076 3051093168552 MainThread DEBUG nova.compute.provider_tree [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:02:59.146 4076 3051093168552 MainThread DEBUG nova.scheduler.client.report [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:02:59.165 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.165 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:02:59.226 4076 3051093168552 MainThread DEBUG nova.compute.utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:02:59.226 4076 3051093171544 GreenThread-73 DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:02:59.226 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:02:59.241 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:59.241 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.241 4076 3051093170136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:02:59.241 4076 3051093170136 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:02:59.241 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:02:59.241 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6A12BD97-581E-4353-8C14-EB27DDF7A97B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112600), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 2, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 2, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:02:59.265 4076 3051113294840 MainThread DEBUG os_win._utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6925852298736572 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:02:59.273 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-af569dde-b533-4e03-8315-82de1eee9c81 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:02:59.285 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-af569dde-b533-4e03-8315-82de1eee9c81 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:02:59.333 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-af569dde-b533-4e03-8315-82de1eee9c81 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:02:59.392 4076 3051093170136 MainThread INFO nova.compute.manager [req-af569dde-b533-4e03-8315-82de1eee9c81 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:02:59.412 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:02:59.412 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Spawning new instance 2021-09-10 17:02:59.444 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:59.444 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.444 4076 3051093168552 MainThread DEBUG nova.block_device [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:02:59.444 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:59.444 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.444 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Received event network-vif-plugged-501733fd-1af5-48d6-aec1-204bd50f155d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Processing event network-vif-plugged-501733fd-1af5-48d6-aec1-204bd50f155d _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Received event network-vif-plugged-d758a788-2d3f-46ad-b1bd-0ee3066bfc4e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:02:59.455 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fb76af40-f285-401f-8ff5-0a4b54c980b6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Processing event network-vif-plugged-d758a788-2d3f-46ad-b1bd-0ee3066bfc4e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:02:59.455 4076 3051093168552 MainThread DEBUG os_win.utils.pathutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-00000056 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:02:59.455 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000056\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:02:59.490 4076 3051093171544 GreenThread-73 DEBUG nova.policy [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:02:59.490 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Resizing VHD C:\openstack\instances\instance-00000056\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:02:59.514 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Using config drive for instance 2021-09-10 17:02:59.564 4076 3051113292728 MainThread INFO nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Using config drive for instance 2021-09-10 17:02:59.646 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Creating config drive at C:\openstack\instances\instance-00000051\configdrive.iso 2021-09-10 17:02:59.685 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000051\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpm7o53wp7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:02:59.988 4076 3051113292728 MainThread INFO nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Creating config drive at C:\openstack\instances\instance-00000055\configdrive.iso 2021-09-10 17:03:00.035 4076 3051113292728 MainThread DEBUG oslo_concurrency.processutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000055\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpppxm1bnt execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:00.208 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000051\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpm7o53wp7" returned: 0 in 0.515s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:00.303 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '42F8EB5A-93B1-497F-B9C9-4DFD191CF509', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19394), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:00.395 4076 3051113292728 MainThread DEBUG oslo_concurrency.processutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000055\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpppxm1bnt" returned: 0 in 0.360s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:00.462 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:00.462 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:00.462 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:00.462 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:03:00.475 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '42F8EB5A-93B1-497F-B9C9-4DFD191CF509', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=127083), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:00.475 4076 3051113294840 MainThread DEBUG os_win._utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.902154445648193 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:00.487 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-453d8c26-d58a-4f4b-bb37-f270ceb3f1c3 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:00.487 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-453d8c26-d58a-4f4b-bb37-f270ceb3f1c3 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:00.553 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-453d8c26-d58a-4f4b-bb37-f270ceb3f1c3 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:00.581 4076 3051093170664 MainThread INFO nova.compute.manager [req-453d8c26-d58a-4f4b-bb37-f270ceb3f1c3 - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:00.722 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Setting boot order for instance: instance-00000051: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:00.722 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Successfully created port: fefdc72a-eece-4cde-a1c2-36a2a40e0912 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:00.779 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:00.806 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '5FD232C9-48DB-4809-A73E-EA6D321C4F9F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14751), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:00.864 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-00000055: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:00.909 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:00.945 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5FD232C9-48DB-4809-A73E-EA6D321C4F9F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120268), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:00.945 4076 3051093170312 MainThread DEBUG os_win._utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16655468940734863 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:00.945 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '2EBFB57D-9F49-47F2-AECF-B388869FF060', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16472), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:00.971 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:00.971 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:00.971 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:00.971 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:00.990 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-a936f0bb-fc60-42b4-bff1-7049df2ea65d - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:00.990 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-a936f0bb-fc60-42b4-bff1-7049df2ea65d - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:01.048 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-a936f0bb-fc60-42b4-bff1-7049df2ea65d - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:01.064 4076 3051093170664 MainThread INFO nova.compute.manager [req-a936f0bb-fc60-42b4-bff1-7049df2ea65d - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:01.081 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:01.081 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:01.081 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:01.081 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:01.081 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2EBFB57D-9F49-47F2-AECF-B388869FF060', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115205), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:01.081 4076 3051113292728 MainThread DEBUG os_win._utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17195725440979004 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:01.111 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-5cdabde7-4bb3-4b7c-a4df-9d197c25e941 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:01.111 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-5cdabde7-4bb3-4b7c-a4df-9d197c25e941 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:01.158 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-5cdabde7-4bb3-4b7c-a4df-9d197c25e941 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:01.177 4076 3051093170664 MainThread INFO nova.compute.manager [req-5cdabde7-4bb3-4b7c-a4df-9d197c25e941 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:01.506 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '164050FD-CBD4-48C6-A6D8-3730B3CDE32C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14915), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:01.627 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:01.627 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:01.627 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:01.627 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] VM Stopped (Lifecycle Event) 2021-09-10 17:03:01.659 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000054\' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF38870210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000054\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)instance-000000545C1D3CDB-ECDB-495C-9A70-A440BF388702%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '164050FD-CBD4-48C6-A6D8-3730B3CDE32C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110766), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\nNot enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)\n\n'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702)", 'ErrorSummaryDescription': "'instance-00000054' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000054.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:01.659 4076 3051113294840 MainThread ERROR nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Failed to change vm state of instance-00000054 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. 2021-09-10 17:03:01.659 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Got request to destroy instance 2021-09-10 17:03:01.659 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-611c21bc-902a-4eb5-9946-5eb12ab8bf6b - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:01.659 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-611c21bc-902a-4eb5-9946-5eb12ab8bf6b - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:01.707 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-611c21bc-902a-4eb5-9946-5eb12ab8bf6b - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:01.729 4076 3051093170664 MainThread INFO nova.compute.manager [req-611c21bc-902a-4eb5-9946-5eb12ab8bf6b - - - - -] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:01.792 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Successfully updated port: fefdc72a-eece-4cde-a1c2-36a2a40e0912 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:01.814 4076 3051093171544 GreenThread-73 DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-953a9285-8fdf-45ca-bec8-d0d9cc154d5d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:01.814 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:01.957 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '6C63B464-6C2C-4C80-9F8B-43D0727D656F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14949), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:02.067 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:02.113 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:02.113 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:02.113 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:02.113 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:02.113 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6C63B464-6C2C-4C80-9F8B-43D0727D656F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119941), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:02.113 4076 3051093170312 MainThread DEBUG os_win._utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.334937572479248 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:02.113 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'C0229CA5-3710-44E4-AE10-CF5F4A1FE4A9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17945), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:02.131 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-c82f77d0-bed1-4339-a26f-813f8250f1dd - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:02.131 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-c82f77d0-bed1-4339-a26f-813f8250f1dd - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:02.191 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-c82f77d0-bed1-4339-a26f-813f8250f1dd - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:02.222 4076 3051093170664 MainThread INFO nova.compute.manager [req-c82f77d0-bed1-4339-a26f-813f8250f1dd - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:02.256 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Received event network-changed-fefdc72a-eece-4cde-a1c2-36a2a40e0912 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:02.256 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Refreshing instance network info cache due to event network-changed-fefdc72a-eece-4cde-a1c2-36a2a40e0912. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:02.256 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C0229CA5-3710-44E4-AE10-CF5F4A1FE4A9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120925), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:02.256 4076 3051113292728 MainThread DEBUG os_win._utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.347029447555542 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:02.271 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:02.271 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:02.271 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:02.271 4076 3051093171192 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:02.284 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-c945e114-cf40-4517-b2a3-9bc9131432fd - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:02.284 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-c945e114-cf40-4517-b2a3-9bc9131432fd - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:02.346 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-c945e114-cf40-4517-b2a3-9bc9131432fd - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:02.379 4076 3051093170664 MainThread INFO nova.compute.manager [req-c945e114-cf40-4517-b2a3-9bc9131432fd - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:02.678 4076 3051093171544 GreenThread-73 DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Updating instance_info_cache with network_info: [{"id": "fefdc72a-eece-4cde-a1c2-36a2a40e0912", "address": "fa:16:3e:57:a5:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfefdc72a-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:02.723 4076 3051093171544 GreenThread-73 DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-953a9285-8fdf-45ca-bec8-d0d9cc154d5d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:02.723 4076 3051093171544 GreenThread-73 DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Instance network_info: |[{"id": "fefdc72a-eece-4cde-a1c2-36a2a40e0912", "address": "fa:16:3e:57:a5:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfefdc72a-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:02.723 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-953a9285-8fdf-45ca-bec8-d0d9cc154d5d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:02.723 4076 3051093169432 MainThread DEBUG nova.network.neutron [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Refreshing network info cache for port fefdc72a-eece-4cde-a1c2-36a2a40e0912 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:02.723 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Preparing to wait for external event network-vif-plugged-fefdc72a-eece-4cde-a1c2-36a2a40e0912 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:02.723 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:02.723 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:02.723 4076 3051093168552 MainThread DEBUG os_win.utils.compute.vmutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-00000056 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:03.154 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '8DDBF7C4-E066-4C8F-82CD-D839B8F30A84', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14790), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:03.298 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '2158B97F-1169-49EB-BA65-910E9F51B177', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=26171), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:03.298 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.298 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:03.314 4076 3051093171192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:03.314 4076 3051093171192 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:03.314 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8DDBF7C4-E066-4C8F-82CD-D839B8F30A84', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117888), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:03.314 4076 3051093170312 MainThread DEBUG os_win._utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.535785675048828 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:03.339 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-1a8ae8e8-7af6-4ca0-9b11-8d4092937225 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:03.339 4076 3051093171192 MainThread DEBUG nova.virt.hyperv.vmops [req-1a8ae8e8-7af6-4ca0-9b11-8d4092937225 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:03.381 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-1a8ae8e8-7af6-4ca0-9b11-8d4092937225 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:03.422 4076 3051093169256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:03.422 4076 3051093169256 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:03.422 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.422 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:03.422 4076 3051093171192 MainThread INFO nova.compute.manager [req-1a8ae8e8-7af6-4ca0-9b11-8d4092937225 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:03.446 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-68061acb-0ae5-413f-8af5-49809f6abe6c - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:03.446 4076 3051093169256 MainThread DEBUG nova.virt.hyperv.vmops [req-68061acb-0ae5-413f-8af5-49809f6abe6c - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:03.462 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2158B97F-1169-49EB-BA65-910E9F51B177', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115511), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:03.462 4076 3051113292728 MainThread DEBUG os_win._utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.552640199661255 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:03.472 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-68061acb-0ae5-413f-8af5-49809f6abe6c - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:03.523 4076 3051093169256 MainThread INFO nova.compute.manager [req-68061acb-0ae5-413f-8af5-49809f6abe6c - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:03.626 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:03.626 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.626 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:03.642 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Successfully changed vm state of instance-00000054 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:03.642 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Successfully changed state of VM instance-00000054 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:03.693 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '519167E1-4285-484C-A88C-9F7C8E7B86F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15920), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:03.724 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:03.818 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job succeeded: {'InstanceID': '519167E1-4285-484C-A88C-9F7C8E7B86F7', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=36912), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:03.818 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.818 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Removing directory: C:\openstack\instances\instance-00000054 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:03.818 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:03.818 4076 3051113294840 MainThread ERROR nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Traceback (most recent call last): 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(*args, **kwargs) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] admin_password, network_info, block_device_info) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(self, *args, **kwds) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] should_plug_vifs=False) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'req_state': req_state}) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] time.sleep(sleep_time) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return f(*args, **kwargs) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return self._wait_for_job(job_path) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] error_desc=err_desc) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During handling of the above exception, another exception occurred: 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Traceback (most recent call last): 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] yield resources 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] accel_info=accel_info) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(*args, **kwargs) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] admin_password, network_info, block_device_info) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(self, *args, **kwds) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] should_plug_vifs=False) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'req_state': req_state}) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] time.sleep(sleep_time) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return f(*args, **kwargs) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return self._wait_for_job(job_path) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] error_desc=err_desc) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:03.818 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:03.842 4076 3051113294840 MainThread INFO nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Terminating instance 2021-09-10 17:03:03.842 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:03.842 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Got request to destroy instance 2021-09-10 17:03:03.853 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:03.853 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.853 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-00000054" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:03.853 4076 3051113294840 MainThread INFO nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:03.853 4076 3051113294840 MainThread DEBUG nova.compute.claims [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:03.853 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:03.988 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:04.004 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:04.060 4076 3051093169432 MainThread DEBUG nova.network.neutron [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Updated VIF entry in instance network info cache for port fefdc72a-eece-4cde-a1c2-36a2a40e0912. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:04.060 4076 3051093169432 MainThread DEBUG nova.network.neutron [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Updating instance_info_cache with network_info: [{"id": "fefdc72a-eece-4cde-a1c2-36a2a40e0912", "address": "fa:16:3e:57:a5:b4", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfefdc72a-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:04.095 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [req-c9cf267e-437f-4fc9-8c7c-7847e55b1969 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-953a9285-8fdf-45ca-bec8-d0d9cc154d5d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:04.224 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:04.254 4076 3051113294840 MainThread ERROR nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Traceback (most recent call last): 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(*args, **kwargs) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] admin_password, network_info, block_device_info) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(self, *args, **kwds) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] should_plug_vifs=False) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'req_state': req_state}) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] time.sleep(sleep_time) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return f(*args, **kwargs) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return self._wait_for_job(job_path) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] error_desc=err_desc) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] During handling of the above exception, another exception occurred: 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Traceback (most recent call last): 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] accel_info=accel_info) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(*args, **kwargs) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] admin_password, network_info, block_device_info) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return function(self, *args, **kwds) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] should_plug_vifs=False) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'req_state': req_state}) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] time.sleep(sleep_time) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self.force_reraise() 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] raise self.value 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return f(*args, **kwargs) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] return self._wait_for_job(job_path) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] error_desc=err_desc) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. 2021-09-10 17:03:04.254 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Build of instance d358307f-112b-4be6-9b2a-08f9ad26205e was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000054' failed to start. Not enough memory in the system to start the virtual machine instance-00000054. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000054' failed to start. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Not enough memory in the system to start the virtual machine instance-00000054 with ram size 1024 megabytes. (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) 'instance-00000054' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5C1D3CDB-ECDB-495C-9A70-A440BF388702) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:04.254 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:04.348 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '0F6D27AF-FDD2-413C-AF66-8825D845ADAD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15599), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:04.490 4076 3051093172072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:04.490 4076 3051093172072 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:04.490 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:04.490 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:04.490 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '20659B3C-9B37-49AB-92F8-24E1A95B1C1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19253), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:04.490 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0F6D27AF-FDD2-413C-AF66-8825D845ADAD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113593), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:04.490 4076 3051093170312 MainThread DEBUG os_win._utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.711000680923462 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:04.520 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-e971e91e-e04b-40ad-9636-1b4b3b585c93 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:04.520 4076 3051093172072 MainThread DEBUG nova.virt.hyperv.vmops [req-e971e91e-e04b-40ad-9636-1b4b3b585c93 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:04.569 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-e971e91e-e04b-40ad-9636-1b4b3b585c93 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:04.599 4076 3051093172072 MainThread INFO nova.compute.manager [req-e971e91e-e04b-40ad-9636-1b4b3b585c93 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:04.611 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:04.611 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:04.611 4076 3051093172072 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:04.611 4076 3051093172072 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:04.645 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '20659B3C-9B37-49AB-92F8-24E1A95B1C1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114977), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:04.645 4076 3051113292728 MainThread DEBUG os_win._utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7364327907562256 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:04.645 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-978b6e87-ab95-4a98-b3cd-d4d8e3bae962 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:04.645 4076 3051093172072 MainThread DEBUG nova.virt.hyperv.vmops [req-978b6e87-ab95-4a98-b3cd-d4d8e3bae962 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:04.693 4076 3051093172072 MainThread DEBUG nova.compute.manager [req-978b6e87-ab95-4a98-b3cd-d4d8e3bae962 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:04.722 4076 3051093172072 MainThread INFO nova.compute.manager [req-978b6e87-ab95-4a98-b3cd-d4d8e3bae962 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:05.505 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '7F48F7EF-E6AD-47C1-93A6-896C94A45D5F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14967), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:05.580 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:05.628 4076 3051113294840 MainThread INFO nova.compute.manager [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d358307f-112b-4be6-9b2a-08f9ad26205e] Took 1.38 seconds to deallocate network for instance. 2021-09-10 17:03:05.659 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:05.659 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:05.659 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:05.659 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:05.659 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7F48F7EF-E6AD-47C1-93A6-896C94A45D5F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114308), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:05.659 4076 3051093170312 MainThread DEBUG os_win._utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.880864143371582 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:05.674 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-38cb4544-41c8-4a2f-8b26-d0fa3c24df55 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:05.674 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-38cb4544-41c8-4a2f-8b26-d0fa3c24df55 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:05.674 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'C36095BB-1D75-4E6E-8575-1D9F5B0E8508', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17738), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:05.727 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-38cb4544-41c8-4a2f-8b26-d0fa3c24df55 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:05.751 4076 3051093171896 MainThread INFO nova.compute.manager [req-38cb4544-41c8-4a2f-8b26-d0fa3c24df55 - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:05.801 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:05.801 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:05.801 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:05.818 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:05.818 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C36095BB-1D75-4E6E-8575-1D9F5B0E8508', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111702), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:05.818 4076 3051113292728 MainThread DEBUG os_win._utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.909280061721802 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:05.841 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-fc006467-56d5-4d5d-a492-20ca53714dae - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:05.841 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-fc006467-56d5-4d5d-a492-20ca53714dae - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:05.885 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-fc006467-56d5-4d5d-a492-20ca53714dae - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:05.913 4076 3051113294840 MainThread INFO nova.scheduler.client.report [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Deleted allocations for instance d358307f-112b-4be6-9b2a-08f9ad26205e 2021-09-10 17:03:05.913 4076 3051093171896 MainThread INFO nova.compute.manager [req-fc006467-56d5-4d5d-a492-20ca53714dae - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:05.973 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-e0b907f3-2e5f-4e44-a734-89fe2900ab1f 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d358307f-112b-4be6-9b2a-08f9ad26205e" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.734s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:06.689 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': 'F9031CE7-ED66-49C8-B9DB-CE5FBF5871FA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14430), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:06.820 4076 3051093169432 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:06.820 4076 3051093169432 MainThread INFO nova.compute.manager [-] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:06.820 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:06.820 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:06.846 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-ea970639-ef9c-4bd1-9f74-c4ca047afd8b - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:06.846 4076 3051093169432 MainThread DEBUG nova.virt.hyperv.vmops [req-ea970639-ef9c-4bd1-9f74-c4ca047afd8b - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:06.846 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000051\' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E0243007610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000051\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)instance-000000516674A5E8-7A7E-46DB-BF1D-8E7E02430076%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9031CE7-ED66-49C8-B9DB-CE5FBF5871FA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113928), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\nNot enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)\n\n'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076)", 'ErrorSummaryDescription': "'instance-00000051' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000051.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:06.846 4076 3051093170312 MainThread ERROR nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Failed to change vm state of instance-00000051 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. 2021-09-10 17:03:06.846 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Got request to destroy instance 2021-09-10 17:03:06.846 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '20FB46D7-5001-4E56-ACE3-A8810A63E6AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21111), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:06.892 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-ea970639-ef9c-4bd1-9f74-c4ca047afd8b - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:06.934 4076 3051093169432 MainThread INFO nova.compute.manager [req-ea970639-ef9c-4bd1-9f74-c4ca047afd8b - - - - -] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:06.988 4076 3051093169432 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:06.988 4076 3051093169432 MainThread INFO nova.compute.manager [-] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] VM Stopped (Lifecycle Event) 2021-09-10 17:03:06.988 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:06.988 4076 3051093171544 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:06.988 4076 3051113292728 MainThread ERROR os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000055\' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000055\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)instance-0000005546E0AE4F-F42C-4F4A-9AA1-EAFC77974F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '20FB46D7-5001-4E56-ACE3-A8810A63E6AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=121883), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\nNot enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)\n\n'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09)", 'ErrorSummaryDescription': "'instance-00000055' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000055.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:06.988 4076 3051113292728 MainThread ERROR nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-00000055 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. 2021-09-10 17:03:06.988 4076 3051113292728 MainThread INFO nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Got request to destroy instance 2021-09-10 17:03:07.032 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-93aaf17c-1eb3-47ed-b228-a1e60c18aa50 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:07.032 4076 3051093169432 MainThread DEBUG nova.virt.hyperv.vmops [req-93aaf17c-1eb3-47ed-b228-a1e60c18aa50 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:07.082 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-93aaf17c-1eb3-47ed-b228-a1e60c18aa50 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:07.123 4076 3051093169432 MainThread INFO nova.compute.manager [req-93aaf17c-1eb3-47ed-b228-a1e60c18aa50 - - - - -] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:08.552 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-0659b99a-8275-42cc-a0ab-2565aab2aeb6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Received event network-vif-plugged-fefdc72a-eece-4cde-a1c2-36a2a40e0912 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:08.552 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [req-0659b99a-8275-42cc-a0ab-2565aab2aeb6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.552 4076 3051093169432 MainThread DEBUG oslo_concurrency.lockutils [req-0659b99a-8275-42cc-a0ab-2565aab2aeb6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:08.552 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-0659b99a-8275-42cc-a0ab-2565aab2aeb6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Processing event network-vif-plugged-fefdc72a-eece-4cde-a1c2-36a2a40e0912 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:08.584 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Using config drive for instance 2021-09-10 17:03:08.737 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:08.737 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.737 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:08.759 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Successfully changed vm state of instance-00000051 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:08.759 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Successfully changed state of VM instance-00000051 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:08.805 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Waiting for WMI job: {'InstanceID': '8ACFB2FD-F7F1-487C-93A4-F8D6FEC766B8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15099), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:08.874 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:08.874 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.874 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:08.892 4076 3051113292728 MainThread DEBUG os_win.utils.compute.vmutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-00000055 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:08.892 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-00000055 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:08.917 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] WMI job succeeded: {'InstanceID': '8ACFB2FD-F7F1-487C-93A4-F8D6FEC766B8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29261), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:08.917 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.917 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Removing directory: C:\openstack\instances\instance-00000051 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:08.917 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:08.917 4076 3051093170312 MainThread ERROR nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Traceback (most recent call last): 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(*args, **kwargs) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] admin_password, network_info, block_device_info) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(self, *args, **kwds) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] should_plug_vifs=False) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'req_state': req_state}) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] time.sleep(sleep_time) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return f(*args, **kwargs) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return self._wait_for_job(job_path) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] error_desc=err_desc) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During handling of the above exception, another exception occurred: 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Traceback (most recent call last): 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] yield resources 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] accel_info=accel_info) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(*args, **kwargs) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] admin_password, network_info, block_device_info) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(self, *args, **kwds) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] should_plug_vifs=False) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'req_state': req_state}) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] time.sleep(sleep_time) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return f(*args, **kwargs) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return self._wait_for_job(job_path) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] error_desc=err_desc) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:08.917 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:08.917 4076 3051093170312 MainThread INFO nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Terminating instance 2021-09-10 17:03:08.917 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:08.917 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Got request to destroy instance 2021-09-10 17:03:08.941 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '647CBA1B-7362-47E1-AD28-CF2D8BBFF83B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20615), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:08.941 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:08.941 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.941 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "instance-00000051" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:08.941 4076 3051093170312 MainThread INFO nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:08.941 4076 3051093170312 MainThread DEBUG nova.compute.claims [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:08.941 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:08.974 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Creating config drive at C:\openstack\instances\instance-00000056\configdrive.iso 2021-09-10 17:03:09.032 4076 3051093168552 MainThread DEBUG oslo_concurrency.processutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000056\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp59bs37n8 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:09.178 4076 3051113292728 MainThread DEBUG os_win.utils.jobutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': '647CBA1B-7362-47E1-AD28-CF2D8BBFF83B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29443), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:09.178 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:09.178 4076 3051113292728 MainThread DEBUG os_win.utils.pathutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-00000055 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:09.178 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:09.178 4076 3051113292728 MainThread ERROR nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Traceback (most recent call last): 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(*args, **kwargs) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(self, *args, **kwds) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] should_plug_vifs=False) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'req_state': req_state}) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] time.sleep(sleep_time) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return f(*args, **kwargs) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return self._wait_for_job(job_path) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] error_desc=err_desc) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During handling of the above exception, another exception occurred: 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Traceback (most recent call last): 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] yield resources 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] accel_info=accel_info) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(*args, **kwargs) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(self, *args, **kwds) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] should_plug_vifs=False) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'req_state': req_state}) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] time.sleep(sleep_time) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return f(*args, **kwargs) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return self._wait_for_job(job_path) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] error_desc=err_desc) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.178 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.178 4076 3051113292728 MainThread INFO nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Terminating instance 2021-09-10 17:03:09.178 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:09.178 4076 3051113292728 MainThread INFO nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Got request to destroy instance 2021-09-10 17:03:09.209 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:09.209 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:09.209 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:09.209 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000055" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:09.209 4076 3051113292728 MainThread INFO nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:09.209 4076 3051113292728 MainThread DEBUG nova.compute.claims [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:09.226 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:09.334 4076 3051093168552 MainThread DEBUG oslo_concurrency.processutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000056\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp59bs37n8" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:09.457 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:09.457 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:09.457 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.515s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:09.457 4076 3051093170312 MainThread ERROR nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Traceback (most recent call last): 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(*args, **kwargs) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(self, *args, **kwds) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] should_plug_vifs=False) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'req_state': req_state}) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] time.sleep(sleep_time) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return f(*args, **kwargs) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return self._wait_for_job(job_path) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] error_desc=err_desc) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] During handling of the above exception, another exception occurred: 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Traceback (most recent call last): 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] accel_info=accel_info) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(*args, **kwargs) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return function(self, *args, **kwds) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] should_plug_vifs=False) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'req_state': req_state}) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] time.sleep(sleep_time) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self.force_reraise() 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] raise self.value 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return f(*args, **kwargs) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] return self._wait_for_job(job_path) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] error_desc=err_desc) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. 2021-09-10 17:03:09.457 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] 2021-09-10 17:03:09.457 4076 3051093170312 MainThread DEBUG nova.compute.utils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:09.457 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Build of instance c4dcbaf6-eadf-45db-9755-790791af2bf7 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000051' failed to start. Not enough memory in the system to start the virtual machine instance-00000051. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000051' failed to start. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Not enough memory in the system to start the virtual machine instance-00000051 with ram size 1024 megabytes. (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) 'instance-00000051' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6674A5E8-7A7E-46DB-BF1D-8E7E02430076) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:09.484 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:09.484 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:09.484 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:09.484 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:09.484 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:09.595 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.hostops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:09.599 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.hostops [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:09.747 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-00000056: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:09.799 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:09.799 4076 3051113292728 MainThread DEBUG nova.compute.provider_tree [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.scheduler.client.report [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:09.830 4076 3051113292728 MainThread ERROR nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Traceback (most recent call last): 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(*args, **kwargs) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(self, *args, **kwds) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] should_plug_vifs=False) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'req_state': req_state}) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] time.sleep(sleep_time) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return f(*args, **kwargs) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return self._wait_for_job(job_path) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] error_desc=err_desc) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] During handling of the above exception, another exception occurred: 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Traceback (most recent call last): 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] accel_info=accel_info) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(*args, **kwargs) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] admin_password, network_info, block_device_info) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return function(self, *args, **kwds) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] should_plug_vifs=False) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'req_state': req_state}) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] time.sleep(sleep_time) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self.force_reraise() 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] raise self.value 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return f(*args, **kwargs) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] return self._wait_for_job(job_path) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] error_desc=err_desc) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. 2021-09-10 17:03:09.830 4076 3051113292728 MainThread TRACE nova.compute.manager [instance: ba57675b-1666-430f-bbaa-84c5ca720900] 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.compute.utils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Build of instance ba57675b-1666-430f-bbaa-84c5ca720900 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000055' failed to start. Not enough memory in the system to start the virtual machine instance-00000055. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000055' failed to start. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Not enough memory in the system to start the virtual machine instance-00000055 with ram size 1024 megabytes. (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) 'instance-00000055' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 46E0AE4F-F42C-4F4A-9AA1-EAFC77974F09) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:09.830 4076 3051113292728 MainThread DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:09.830 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '981325A9-CCCB-40C6-B126-23F52CCAED8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16766), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:09.972 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:09.972 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:09.972 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:09.972 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:09.989 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '981325A9-CCCB-40C6-B126-23F52CCAED8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114556), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:09.989 4076 3051093168552 MainThread DEBUG os_win._utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1898951530456543 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:09.989 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e43160a1-56f4-42b8-9e4b-2e3220ea6053 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:09.989 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-e43160a1-56f4-42b8-9e4b-2e3220ea6053 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:10.049 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-e43160a1-56f4-42b8-9e4b-2e3220ea6053 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:10.081 4076 3051055217304 MainThread INFO nova.compute.manager [req-e43160a1-56f4-42b8-9e4b-2e3220ea6053 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:11.034 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '9DF73209-C361-4CFC-98EF-CF6A388C8277', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15138), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:11.158 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:11.158 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:11.158 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:11.158 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:11.181 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9DF73209-C361-4CFC-98EF-CF6A388C8277', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110605), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:11.181 4076 3051093168552 MainThread DEBUG os_win._utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3820545673370361 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:11.189 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-09b54032-7e89-476b-8258-9c554b56167f - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:11.189 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-09b54032-7e89-476b-8258-9c554b56167f - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:11.224 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-09b54032-7e89-476b-8258-9c554b56167f - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:11.255 4076 3051055217304 MainThread INFO nova.compute.manager [req-09b54032-7e89-476b-8258-9c554b56167f - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:11.565 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:11.615 4076 3051093170312 MainThread INFO nova.compute.manager [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] [instance: c4dcbaf6-eadf-45db-9755-790791af2bf7] Took 2.12 seconds to deallocate network for instance. 2021-09-10 17:03:11.724 4076 3051113292728 MainThread DEBUG nova.network.neutron [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:11.787 4076 3051113292728 MainThread INFO nova.compute.manager [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: ba57675b-1666-430f-bbaa-84c5ca720900] Took 1.95 seconds to deallocate network for instance. 2021-09-10 17:03:11.845 4076 3051093170312 MainThread INFO nova.scheduler.client.report [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Deleted allocations for instance c4dcbaf6-eadf-45db-9755-790791af2bf7 2021-09-10 17:03:11.927 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-21856b11-64e3-4027-a64e-15120ccd3f25 0063b05acec342499daf8129dc0ca365 5d6026a33839407f9f68cb295fbb6928 - default default] Lock "c4dcbaf6-eadf-45db-9755-790791af2bf7" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.922s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:12.066 4076 3051113292728 MainThread INFO nova.scheduler.client.report [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance ba57675b-1666-430f-bbaa-84c5ca720900 2021-09-10 17:03:12.136 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-52e3f898-b55b-4d09-9d87-cc99d15ae661 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "ba57675b-1666-430f-bbaa-84c5ca720900" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 32.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:12.206 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '962A6732-621E-48FD-BBF7-44516DD4F9F1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15565), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:12.334 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:12.334 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:12.334 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:12.334 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:12.334 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '962A6732-621E-48FD-BBF7-44516DD4F9F1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112339), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:12.359 4076 3051093168552 MainThread DEBUG os_win._utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5608696937561035 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:12.392 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2bab87fc-38e1-4593-902e-da9b07bcbc41 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:12.392 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-2bab87fc-38e1-4593-902e-da9b07bcbc41 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:12.440 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-2bab87fc-38e1-4593-902e-da9b07bcbc41 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:12.489 4076 3051093170312 MainThread INFO nova.compute.manager [req-2bab87fc-38e1-4593-902e-da9b07bcbc41 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:13.377 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '14A54F7C-2412-4669-9B45-1E1EC28A0D60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14870), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:13.517 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:13.517 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:13.517 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:13.517 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:13.517 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '14A54F7C-2412-4669-9B45-1E1EC28A0D60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110782), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:13.517 4076 3051093168552 MainThread DEBUG os_win._utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.718308448791504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:13.561 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b24270b5-742b-45b9-b7b6-0fae74e9ad5a - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:13.561 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-b24270b5-742b-45b9-b7b6-0fae74e9ad5a - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:13.603 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b24270b5-742b-45b9-b7b6-0fae74e9ad5a - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:13.662 4076 3051093170312 MainThread INFO nova.compute.manager [req-b24270b5-742b-45b9-b7b6-0fae74e9ad5a - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:14.536 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '2770107B-7D47-4857-A7B8-578251B14E9A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14698), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:14.677 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:14.677 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:14.677 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:14.677 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:14.677 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2770107B-7D47-4857-A7B8-578251B14E9A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108616), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:14.677 4076 3051093168552 MainThread DEBUG os_win._utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8780622482299805 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:14.708 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-15156a6f-880a-4b3e-961f-2a876ba4a86d - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:14.708 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-15156a6f-880a-4b3e-961f-2a876ba4a86d - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:14.755 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-15156a6f-880a-4b3e-961f-2a876ba4a86d - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:14.769 4076 3051093170312 MainThread INFO nova.compute.manager [req-15156a6f-880a-4b3e-961f-2a876ba4a86d - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:15.712 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '5A00F03F-4D97-4287-85FD-D57F2F67925B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15373), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:15.849 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:15.849 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:15.849 4076 3051093171192 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:15.849 4076 3051093171192 MainThread INFO nova.compute.manager [-] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:15.849 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000056\' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F0910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000056\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)instance-000000569EE17F03-B562-4B0E-8CAD-E7F99A942F09%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5A00F03F-4D97-4287-85FD-D57F2F67925B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110687), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\nNot enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)\n\n'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09)", 'ErrorSummaryDescription': "'instance-00000056' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000056.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:15.849 4076 3051093168552 MainThread ERROR nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-00000056 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. 2021-09-10 17:03:15.849 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Got request to destroy instance 2021-09-10 17:03:15.877 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-d1bd3fff-dbdd-4797-be85-edc22b716a38 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:15.877 4076 3051093171192 MainThread DEBUG nova.virt.hyperv.vmops [req-d1bd3fff-dbdd-4797-be85-edc22b716a38 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:15.927 4076 3051093171192 MainThread DEBUG nova.compute.manager [req-d1bd3fff-dbdd-4797-be85-edc22b716a38 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:15.958 4076 3051093171192 MainThread INFO nova.compute.manager [req-d1bd3fff-dbdd-4797-be85-edc22b716a38 - - - - -] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:17.661 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:17.688 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:17.688 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:17.696 4076 3051093168552 MainThread DEBUG os_win.utils.compute.vmutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-00000056 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:17.696 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-00000056 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:17.745 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '00B92563-8714-4639-B8A0-EB596CF77B72', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14283), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:17.863 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': '00B92563-8714-4639-B8A0-EB596CF77B72', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28873), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:17.863 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:17.863 4076 3051093168552 MainThread DEBUG os_win.utils.pathutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-00000056 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:17.881 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:17.881 4076 3051093168552 MainThread ERROR nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Traceback (most recent call last): 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(*args, **kwargs) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] admin_password, network_info, block_device_info) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(self, *args, **kwds) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] should_plug_vifs=False) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'req_state': req_state}) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] time.sleep(sleep_time) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return f(*args, **kwargs) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return self._wait_for_job(job_path) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] error_desc=err_desc) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During handling of the above exception, another exception occurred: 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Traceback (most recent call last): 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] yield resources 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] accel_info=accel_info) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(*args, **kwargs) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] admin_password, network_info, block_device_info) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(self, *args, **kwds) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] should_plug_vifs=False) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'req_state': req_state}) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] time.sleep(sleep_time) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return f(*args, **kwargs) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return self._wait_for_job(job_path) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] error_desc=err_desc) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:17.881 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:17.881 4076 3051093168552 MainThread INFO nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Terminating instance 2021-09-10 17:03:17.881 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:17.881 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Got request to destroy instance 2021-09-10 17:03:17.896 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:17.896 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:17.896 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000056" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:17.896 4076 3051093168552 MainThread INFO nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:17.896 4076 3051093168552 MainThread DEBUG nova.compute.claims [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:17.896 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:18.048 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:18.069 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:18.260 4076 3051093168552 MainThread DEBUG nova.compute.provider_tree [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:18.280 4076 3051093168552 MainThread DEBUG nova.scheduler.client.report [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:18.296 4076 3051093168552 MainThread ERROR nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Traceback (most recent call last): 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(*args, **kwargs) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] admin_password, network_info, block_device_info) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(self, *args, **kwds) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] should_plug_vifs=False) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'req_state': req_state}) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] time.sleep(sleep_time) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return f(*args, **kwargs) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return self._wait_for_job(job_path) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] error_desc=err_desc) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] During handling of the above exception, another exception occurred: 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Traceback (most recent call last): 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] accel_info=accel_info) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(*args, **kwargs) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] admin_password, network_info, block_device_info) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return function(self, *args, **kwds) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] should_plug_vifs=False) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'req_state': req_state}) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] time.sleep(sleep_time) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self.force_reraise() 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] raise self.value 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return f(*args, **kwargs) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] return self._wait_for_job(job_path) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] error_desc=err_desc) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. 2021-09-10 17:03:18.296 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.compute.utils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Build of instance 953a9285-8fdf-45ca-bec8-d0d9cc154d5d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000056' failed to start. Not enough memory in the system to start the virtual machine instance-00000056. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000056' failed to start. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Not enough memory in the system to start the virtual machine instance-00000056 with ram size 1024 megabytes. (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) 'instance-00000056' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9EE17F03-B562-4B0E-8CAD-E7F99A942F09) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:18.296 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:20.680 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:20.704 4076 3051093168552 MainThread INFO nova.compute.manager [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 953a9285-8fdf-45ca-bec8-d0d9cc154d5d] Took 2.41 seconds to deallocate network for instance. 2021-09-10 17:03:20.927 4076 3051093168552 MainThread INFO nova.scheduler.client.report [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance 953a9285-8fdf-45ca-bec8-d0d9cc154d5d 2021-09-10 17:03:20.974 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-1552972b-5777-4936-acc5-e086d2344707 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "953a9285-8fdf-45ca-bec8-d0d9cc154d5d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 22.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:23.528 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "ce937990-b827-4d67-85b2-7c454a6b252b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:23.560 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:23.662 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:23.662 4076 3051093170136 MainThread DEBUG nova.virt.hardware [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:23.662 4076 3051093170136 MainThread INFO nova.compute.claims [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Claim successful on node n-h1-808474-1 2021-09-10 17:03:23.802 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:23.818 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:24.006 4076 3051093170136 MainThread DEBUG nova.compute.provider_tree [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:24.032 4076 3051093170136 MainThread DEBUG nova.scheduler.client.report [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:24.032 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:24.032 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:24.099 4076 3051093170136 MainThread DEBUG nova.compute.utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:24.099 4076 3051093168552 GreenThread-74 DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not allocating networking since 'none' was specified. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1752 2021-09-10 17:03:24.138 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:24.302 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:24.302 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Spawning new instance 2021-09-10 17:03:24.329 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:24.329 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:24.329 4076 3051093170136 MainThread DEBUG nova.block_device [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:24.334 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:24.334 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:24.334 4076 3051093170136 MainThread DEBUG os_win.utils.pathutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Creating directory: C:\openstack\instances\instance-00000058 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:24.334 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000058\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:24.364 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Resizing VHD C:\openstack\instances\instance-00000058\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:24.377 4076 3051093170136 MainThread DEBUG os_win.utils.compute.vmutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Creating VM instance-00000058 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:25.490 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Using config drive for instance 2021-09-10 17:03:25.563 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Creating config drive at C:\openstack\instances\instance-00000058\configdrive.iso 2021-09-10 17:03:25.614 4076 3051093170136 MainThread DEBUG oslo_concurrency.processutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000058\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpw9z_04f5 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:25.896 4076 3051093170136 MainThread DEBUG oslo_concurrency.processutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000058\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpw9z_04f5" returned: 0 in 0.281s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:26.266 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Setting boot order for instance: instance-00000058: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:26.318 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:26.348 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': '3E4A9EE2-6FD2-47E9-89E8-4E54EF4AAFE3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18307), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:26.464 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:26.464 4076 3051093172072 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:26.464 4076 3051093169432 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:26.475 4076 3051093169432 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:26.496 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3E4A9EE2-6FD2-47E9-89E8-4E54EF4AAFE3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=97421), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:26.504 4076 3051093170136 MainThread DEBUG os_win._utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18596529960632324 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:26.504 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-78a07f18-26a4-406f-9590-fb6e4fff3a60 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:26.504 4076 3051093169432 MainThread DEBUG nova.virt.hyperv.vmops [req-78a07f18-26a4-406f-9590-fb6e4fff3a60 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:26.552 4076 3051093169432 MainThread DEBUG nova.compute.manager [req-78a07f18-26a4-406f-9590-fb6e4fff3a60 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:26.594 4076 3051093169432 MainThread INFO nova.compute.manager [req-78a07f18-26a4-406f-9590-fb6e4fff3a60 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:26.983 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:26.990 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:27.114 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:27.114 4076 3051093171896 MainThread DEBUG nova.virt.hardware [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:27.114 4076 3051093171896 MainThread INFO nova.compute.claims [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Claim successful on node n-h1-808474-1 2021-09-10 17:03:27.236 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.hostops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:27.271 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.hostops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:27.483 4076 3051093171896 MainThread DEBUG nova.compute.provider_tree [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:27.486 4076 3051093171896 MainThread DEBUG nova.scheduler.client.report [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:27.486 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:27.505 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "749db01c-d4b1-4eea-aa48-d7dd148b27f7" acquired by "nova.compute.manager.ComputeManager._validate_instance_group_policy.._do_validation" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:27.535 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "749db01c-d4b1-4eea-aa48-d7dd148b27f7" released by "nova.compute.manager.ComputeManager._validate_instance_group_policy.._do_validation" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:27.535 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:27.535 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': '69213C14-DA9A-4510-AA90-979337FA136E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17317), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:27.621 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:27.631 4076 3051093168552 GreenThread-75 DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:27.631 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:27.646 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:27.646 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:27.646 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:27.646 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:27.675 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-6a1b8031-3ceb-41ac-b7f2-6a885d8189ab - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:27.675 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-6a1b8031-3ceb-41ac-b7f2-6a885d8189ab - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:27.693 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '69213C14-DA9A-4510-AA90-979337FA136E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=90383), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:27.693 4076 3051093170136 MainThread DEBUG os_win._utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3757050037384033 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:27.733 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-6a1b8031-3ceb-41ac-b7f2-6a885d8189ab - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:27.755 4076 3051093170664 MainThread INFO nova.compute.manager [req-6a1b8031-3ceb-41ac-b7f2-6a885d8189ab - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:27.782 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:27.798 4076 3051093171896 MainThread INFO nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Spawning new instance 2021-09-10 17:03:27.798 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG nova.block_device [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG os_win.utils.pathutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Creating directory: C:\openstack\instances\instance-00000057 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:27.813 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000057\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:27.828 4076 3051093168552 GreenThread-75 DEBUG nova.policy [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7f13c7f24aed435daa5c4c3caf5d68c9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f7a5fed42ca440bb659840e05ce1f95', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:27.847 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Resizing VHD C:\openstack\instances\instance-00000057\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:28.735 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': 'DC7AD719-376F-4A95-9CEE-13E96A121B0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16257), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:28.850 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:28.850 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:28.850 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:28.850 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:28.882 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7265fa65-3ea6-44c9-a8ca-4ce6da3c878c - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:28.882 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7265fa65-3ea6-44c9-a8ca-4ce6da3c878c - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:28.882 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DC7AD719-376F-4A95-9CEE-13E96A121B0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=95593), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:28.882 4076 3051093170136 MainThread DEBUG os_win._utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5642147064208984 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:28.927 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7265fa65-3ea6-44c9-a8ca-4ce6da3c878c - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:28.957 4076 3051093170664 MainThread INFO nova.compute.manager [req-7265fa65-3ea6-44c9-a8ca-4ce6da3c878c - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:29.016 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Successfully created port: e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:29.912 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': 'E546BDA9-E1D6-4816-AFE7-DDA231F82C16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14858), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:30.014 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:30.014 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:30.014 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:30.014 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:30.044 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-d26d5dc9-d0c4-4b50-a802-612cb80bc56b - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:30.044 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-d26d5dc9-d0c4-4b50-a802-612cb80bc56b - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:30.044 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E546BDA9-E1D6-4816-AFE7-DDA231F82C16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=90412), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:30.052 4076 3051093170136 MainThread DEBUG os_win._utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7344419956207275 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:30.079 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-d26d5dc9-d0c4-4b50-a802-612cb80bc56b - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:30.115 4076 3051093170664 MainThread INFO nova.compute.manager [req-d26d5dc9-d0c4-4b50-a802-612cb80bc56b - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:30.493 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Successfully updated port: e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:30.511 4076 3051093168552 GreenThread-75 DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Acquired lock "refresh_cache-f817e731-74c6-4c6f-8109-5984539d364a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:30.511 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:30.661 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:30.954 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Received event network-changed-e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:30.954 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Refreshing instance network info cache due to event network-changed-e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:31.072 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': 'CEB0FB63-DC74-436D-8BAD-4AC842FF9716', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14559), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:31.182 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:31.198 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:31.198 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:31.198 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:31.214 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CEB0FB63-DC74-436D-8BAD-4AC842FF9716', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=89769), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:31.214 4076 3051093170136 MainThread DEBUG os_win._utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.89674973487854 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:31.221 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-2765d86c-a17f-4f55-b9e6-14137a707385 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:31.221 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-2765d86c-a17f-4f55-b9e6-14137a707385 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:31.235 4076 3051093168552 GreenThread-75 DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Updating instance_info_cache with network_info: [{"id": "e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba", "address": "fa:16:3e:72:07:35", "network": {"id": "531a14a9-553f-4e9e-bbc0-ffeae3b418e3", "bridge": null, "label": "tempest-ServerGroupTestJSON-894972691-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3f7a5fed42ca440bb659840e05ce1f95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape1e84cc7-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:31.252 4076 3051093168552 GreenThread-75 DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Releasing lock "refresh_cache-f817e731-74c6-4c6f-8109-5984539d364a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:31.252 4076 3051093168552 GreenThread-75 DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Instance network_info: |[{"id": "e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba", "address": "fa:16:3e:72:07:35", "network": {"id": "531a14a9-553f-4e9e-bbc0-ffeae3b418e3", "bridge": null, "label": "tempest-ServerGroupTestJSON-894972691-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3f7a5fed42ca440bb659840e05ce1f95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape1e84cc7-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:31.252 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Preparing to wait for external event network-vif-plugged-e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:31.252 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:31.268 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:31.268 4076 3051093171896 MainThread DEBUG os_win.utils.compute.vmutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Creating VM instance-00000057 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:31.268 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f817e731-74c6-4c6f-8109-5984539d364a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:31.268 4076 3051093170664 MainThread DEBUG nova.network.neutron [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Refreshing network info cache for port e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:31.268 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-2765d86c-a17f-4f55-b9e6-14137a707385 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:31.301 4076 3051055217128 MainThread INFO nova.compute.manager [req-2765d86c-a17f-4f55-b9e6-14137a707385 - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:31.755 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:31.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:31.786 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:31.833 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:31.892 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:31.931 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:31.931 4076 3051055217128 MainThread DEBUG nova.virt.hardware [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:31.931 4076 3051055217128 MainThread INFO nova.compute.claims [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Claim successful on node n-h1-808474-1 2021-09-10 17:03:32.087 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:32.097 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:32.160 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:32.239 4076 3051093170664 MainThread DEBUG nova.network.neutron [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Updated VIF entry in instance network info cache for port e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:32.239 4076 3051093170664 MainThread DEBUG nova.network.neutron [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Updating instance_info_cache with network_info: [{"id": "e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba", "address": "fa:16:3e:72:07:35", "network": {"id": "531a14a9-553f-4e9e-bbc0-ffeae3b418e3", "bridge": null, "label": "tempest-ServerGroupTestJSON-894972691-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "3f7a5fed42ca440bb659840e05ce1f95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape1e84cc7-ad", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:32.252 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': 'BF5045E1-4098-451D-A320-5DD0C7C5F6EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19728), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:32.252 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-8c019dd3-5901-49a9-8a39-1283df4cf0c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f817e731-74c6-4c6f-8109-5984539d364a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:32.331 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:32.349 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:32.349 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:32.349 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:32.369 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:32.369 4076 3051055218008 MainThread DEBUG nova.virt.hardware [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:32.369 4076 3051055218008 MainThread INFO nova.compute.claims [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Claim successful on node n-h1-808474-1 2021-09-10 17:03:32.380 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:32.380 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] VM Stopped (Lifecycle Event) 2021-09-10 17:03:32.380 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:32.380 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:32.423 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000058\' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000058\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)instance-000000584719B7E4-CE52-41CA-B836-5D37559F66BF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BF5045E1-4098-451D-A320-5DD0C7C5F6EF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=92071), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\nNot enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)\n\n'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF)", 'ErrorSummaryDescription': "'instance-00000058' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000058.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:32.423 4076 3051093170136 MainThread ERROR nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Failed to change vm state of instance-00000058 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. 2021-09-10 17:03:32.423 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Got request to destroy instance 2021-09-10 17:03:32.481 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:32.481 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-eed610af-fb12-4b74-bccd-a3723b80b4fb - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:32.481 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-eed610af-fb12-4b74-bccd-a3723b80b4fb - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:32.481 4076 3051093168552 GreenThread-76 DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:32.491 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:32.491 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:32.518 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:32.536 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:32.552 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-eed610af-fb12-4b74-bccd-a3723b80b4fb - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:32.573 4076 3051093170312 MainThread INFO nova.compute.manager [req-eed610af-fb12-4b74-bccd-a3723b80b4fb - - - - -] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:32.661 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:32.676 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Spawning new instance 2021-09-10 17:03:32.676 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:32.676 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:32.676 4076 3051055217128 MainThread DEBUG nova.block_device [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:32.693 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:32.693 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:32.693 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Creating directory: C:\openstack\instances\instance-00000059 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:32.693 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000059\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:32.722 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Resizing VHD C:\openstack\instances\instance-00000059\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:32.771 4076 3051055218008 MainThread DEBUG nova.compute.provider_tree [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:32.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:32.786 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:03:32.786 4076 3051055218008 MainThread DEBUG nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:32.803 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:32.803 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:32.914 4076 3051093168552 GreenThread-76 DEBUG nova.policy [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f602a288fe3b4ea0bf8af5b656e8681a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7055b5c60f1f45d6b0dd2b8a9a6f7f0a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:32.931 4076 3051055218008 MainThread DEBUG nova.compute.utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:32.931 4076 3051093170664 GreenThread-77 DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:32.931 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:32.985 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:33.177 4076 3051093170664 GreenThread-77 DEBUG nova.policy [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f602a288fe3b4ea0bf8af5b656e8681a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7055b5c60f1f45d6b0dd2b8a9a6f7f0a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:33.192 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:33.192 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:03:33.257 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:33.257 4076 3051055218008 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Spawning new instance 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG nova.block_device [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG os_win.utils.pathutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Creating directory: C:\openstack\instances\instance-0000005a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:33.277 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000005a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:33.319 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Resizing VHD C:\openstack\instances\instance-0000005a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:34.230 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:34.240 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:34.240 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:34.240 4076 3051093170136 MainThread DEBUG os_win.utils.compute.vmutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Successfully changed vm state of instance-00000058 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:34.240 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Successfully changed state of VM instance-00000058 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:34.287 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for WMI job: {'InstanceID': '3165C011-3F31-480A-9730-935F480DFB9A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13452), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:34.417 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] WMI job succeeded: {'InstanceID': '3165C011-3F31-480A-9730-935F480DFB9A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28317), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:34.417 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:34.417 4076 3051093170136 MainThread DEBUG os_win.utils.pathutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Removing directory: C:\openstack\instances\instance-00000058 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:34.417 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:34.417 4076 3051093170136 MainThread ERROR nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Traceback (most recent call last): 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(*args, **kwargs) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] admin_password, network_info, block_device_info) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(self, *args, **kwds) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] should_plug_vifs=False) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'req_state': req_state}) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] time.sleep(sleep_time) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return f(*args, **kwargs) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return self._wait_for_job(job_path) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] error_desc=err_desc) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During handling of the above exception, another exception occurred: 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Traceback (most recent call last): 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] yield resources 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] accel_info=accel_info) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(*args, **kwargs) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] admin_password, network_info, block_device_info) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(self, *args, **kwds) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] should_plug_vifs=False) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'req_state': req_state}) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] time.sleep(sleep_time) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return f(*args, **kwargs) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return self._wait_for_job(job_path) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] error_desc=err_desc) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:34.417 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:34.432 4076 3051093170136 MainThread INFO nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Terminating instance 2021-09-10 17:03:34.432 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Acquired lock "refresh_cache-ce937990-b827-4d67-85b2-7c454a6b252b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:34.432 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:34.488 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:34.704 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:34.704 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:34.719 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:34.719 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Releasing lock "refresh_cache-ce937990-b827-4d67-85b2-7c454a6b252b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:34.736 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Got request to destroy instance 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "instance-00000058" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:34.736 4076 3051093170136 MainThread INFO nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG oslo.service.loopingcall [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.._deallocate_network_with_retries to return. func c:\python37\lib\site-packages\oslo_service\loopingcall.py:435 2021-09-10 17:03:34.736 4076 3051055216776 GreenThread-78 DEBUG nova.compute.manager [-] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:03:34.736 4076 3051093170136 MainThread DEBUG nova.compute.claims [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:34.763 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:34.898 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Successfully created port: fc5f4b1e-56fb-453c-b480-f6987341eda9 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:34.907 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:34.907 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:35.116 4076 3051093170136 MainThread DEBUG nova.compute.provider_tree [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG nova.scheduler.client.report [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:35.144 4076 3051093170136 MainThread ERROR nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Traceback (most recent call last): 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(*args, **kwargs) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] admin_password, network_info, block_device_info) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(self, *args, **kwds) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] should_plug_vifs=False) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'req_state': req_state}) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] time.sleep(sleep_time) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return f(*args, **kwargs) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return self._wait_for_job(job_path) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] error_desc=err_desc) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] During handling of the above exception, another exception occurred: 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Traceback (most recent call last): 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] accel_info=accel_info) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(*args, **kwargs) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] admin_password, network_info, block_device_info) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return function(self, *args, **kwds) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] should_plug_vifs=False) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'req_state': req_state}) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] time.sleep(sleep_time) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self.force_reraise() 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] raise self.value 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return f(*args, **kwargs) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] return self._wait_for_job(job_path) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] error_desc=err_desc) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. 2021-09-10 17:03:35.144 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: ce937990-b827-4d67-85b2-7c454a6b252b] 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG nova.compute.utils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Build of instance ce937990-b827-4d67-85b2-7c454a6b252b was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000058' failed to start. Not enough memory in the system to start the virtual machine instance-00000058. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000058' failed to start. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Not enough memory in the system to start the virtual machine instance-00000058 with ram size 1024 megabytes. (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) 'instance-00000058' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 4719B7E4-CE52-41CA-B836-5D37559F66BF) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Acquired lock "refresh_cache-ce937990-b827-4d67-85b2-7c454a6b252b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:35.144 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:35.240 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:35.703 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:35.719 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Releasing lock "refresh_cache-ce937990-b827-4d67-85b2-7c454a6b252b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:35.719 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:35.719 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] [instance: ce937990-b827-4d67-85b2-7c454a6b252b] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:03:35.860 4076 3051093170136 MainThread INFO nova.scheduler.client.report [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Deleted allocations for instance ce937990-b827-4d67-85b2-7c454a6b252b 2021-09-10 17:03:35.929 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a8dcf3b-4b50-49c5-9d06-d323d71793f6 905e3c6ad96d46c19a0c053a75b0be3f c076dc2ea49744da8488050f198edf70 - default default] Lock "ce937990-b827-4d67-85b2-7c454a6b252b" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 12.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:36.006 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:36.006 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:36.188 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Successfully created port: 9ea2b698-897f-4c4a-80d8-f74194d1fead _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:37.616 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Successfully updated port: fc5f4b1e-56fb-453c-b480-f6987341eda9 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:37.662 4076 3051093168552 GreenThread-76 DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Acquired lock "refresh_cache-4a2b3b75-4c02-4601-b40f-1e37d8b022d7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:37.662 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:37.786 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:37.854 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Received event network-changed-fc5f4b1e-56fb-453c-b480-f6987341eda9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:37.854 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Refreshing instance network info cache due to event network-changed-fc5f4b1e-56fb-453c-b480-f6987341eda9. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:38.104 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Successfully updated port: 9ea2b698-897f-4c4a-80d8-f74194d1fead _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:38.126 4076 3051093170664 GreenThread-77 DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Acquired lock "refresh_cache-317552ba-8b42-4d4f-a84c-6bea20b13adb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:38.126 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:38.350 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:38.490 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-6cbf936e-b352-47eb-8433-20544ad12e7b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Received event network-vif-plugged-e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:38.490 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-6cbf936e-b352-47eb-8433-20544ad12e7b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:38.490 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-6cbf936e-b352-47eb-8433-20544ad12e7b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:38.490 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-6cbf936e-b352-47eb-8433-20544ad12e7b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Processing event network-vif-plugged-e1e84cc7-ad49-4f8c-aad9-dd274e7f2bba _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:38.528 4076 3051093171896 MainThread INFO nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Using config drive for instance 2021-09-10 17:03:38.709 4076 3051093168552 GreenThread-76 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Updating instance_info_cache with network_info: [{"id": "fc5f4b1e-56fb-453c-b480-f6987341eda9", "address": "fa:16:3e:32:80:01", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfc5f4b1e-56", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:38.750 4076 3051093168552 GreenThread-76 DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Releasing lock "refresh_cache-4a2b3b75-4c02-4601-b40f-1e37d8b022d7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:38.766 4076 3051093168552 GreenThread-76 DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Instance network_info: |[{"id": "fc5f4b1e-56fb-453c-b480-f6987341eda9", "address": "fa:16:3e:32:80:01", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfc5f4b1e-56", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:38.766 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Preparing to wait for external event network-vif-plugged-fc5f4b1e-56fb-453c-b480-f6987341eda9 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:38.766 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:38.766 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:38.766 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Creating VM instance-00000059 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:38.766 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4a2b3b75-4c02-4601-b40f-1e37d8b022d7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:38.766 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Refreshing network info cache for port fc5f4b1e-56fb-453c-b480-f6987341eda9 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:38.782 4076 3051093171896 MainThread INFO nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Creating config drive at C:\openstack\instances\instance-00000057\configdrive.iso 2021-09-10 17:03:38.818 4076 3051093171896 MainThread DEBUG oslo_concurrency.processutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000057\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpps3z0ap1 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:38.951 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:38.978 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:39.123 4076 3051093171896 MainThread DEBUG oslo_concurrency.processutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000057\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpps3z0ap1" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:39.252 4076 3051093170664 GreenThread-77 DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Updating instance_info_cache with network_info: [{"id": "9ea2b698-897f-4c4a-80d8-f74194d1fead", "address": "fa:16:3e:bb:27:70", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ea2b698-89", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:39.284 4076 3051093170664 GreenThread-77 DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Releasing lock "refresh_cache-317552ba-8b42-4d4f-a84c-6bea20b13adb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:39.284 4076 3051093170664 GreenThread-77 DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Instance network_info: |[{"id": "9ea2b698-897f-4c4a-80d8-f74194d1fead", "address": "fa:16:3e:bb:27:70", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ea2b698-89", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:39.284 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Preparing to wait for external event network-vif-plugged-9ea2b698-897f-4c4a-80d8-f74194d1fead prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:39.284 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:39.284 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:39.284 4076 3051055218008 MainThread DEBUG os_win.utils.compute.vmutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Creating VM instance-0000005a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:39.517 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Setting boot order for instance: instance-00000057: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:39.569 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:39.646 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '93B77D4C-045B-40AE-AB30-DC0FA3A9AC73', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15356), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:39.673 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:39.771 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:39.771 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:39.771 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:39.771 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:39.787 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '93B77D4C-045B-40AE-AB30-DC0FA3A9AC73', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115019), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:39.787 4076 3051093171896 MainThread DEBUG os_win._utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.21813702583312988 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:39.803 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-ecc10963-80fb-4b3e-b387-741029874357 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:39.803 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-ecc10963-80fb-4b3e-b387-741029874357 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:39.846 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-ecc10963-80fb-4b3e-b387-741029874357 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:39.879 4076 3051093170664 MainThread INFO nova.compute.manager [req-ecc10963-80fb-4b3e-b387-741029874357 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:40.235 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Updated VIF entry in instance network info cache for port fc5f4b1e-56fb-453c-b480-f6987341eda9. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:40.235 4076 3051055217480 MainThread DEBUG nova.network.neutron [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Updating instance_info_cache with network_info: [{"id": "fc5f4b1e-56fb-453c-b480-f6987341eda9", "address": "fa:16:3e:32:80:01", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapfc5f4b1e-56", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:40.235 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:40.235 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [req-19e4e645-7b72-4b9a-9830-eeb57ec6b47a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4a2b3b75-4c02-4601-b40f-1e37d8b022d7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:40.301 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Received event network-changed-9ea2b698-897f-4c4a-80d8-f74194d1fead external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:40.301 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Refreshing instance network info cache due to event network-changed-9ea2b698-897f-4c4a-80d8-f74194d1fead. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:40.301 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-317552ba-8b42-4d4f-a84c-6bea20b13adb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:40.301 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Refreshing network info cache for port 9ea2b698-897f-4c4a-80d8-f74194d1fead _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:40.797 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:40.797 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:40.797 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:03:40.797 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:40.814 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:40.814 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:40.814 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:03:40.814 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:40.831 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': 'B3B427D2-5EAA-408B-823C-66BA034930CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16837), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:40.847 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:40.969 4076 3051093170664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:40.969 4076 3051093170664 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:40.969 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:40.969 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:40.969 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B3B427D2-5EAA-408B-823C-66BA034930CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122794), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:40.969 4076 3051093171896 MainThread DEBUG os_win._utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4009079933166504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:41.001 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-175d828e-5740-49e1-a748-0d92d1f9787f - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:41.001 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-175d828e-5740-49e1-a748-0d92d1f9787f - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:41.037 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-175d828e-5740-49e1-a748-0d92d1f9787f - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:41.079 4076 3051093170664 MainThread INFO nova.compute.manager [req-175d828e-5740-49e1-a748-0d92d1f9787f - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:41.079 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Updated VIF entry in instance network info cache for port 9ea2b698-897f-4c4a-80d8-f74194d1fead. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:41.079 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Updating instance_info_cache with network_info: [{"id": "9ea2b698-897f-4c4a-80d8-f74194d1fead", "address": "fa:16:3e:bb:27:70", "network": {"id": "51960055-efb4-4c4f-8674-2e7a379abd57", "bridge": null, "label": "tempest-ListServersNegativeTestJSON-278608844-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "7055b5c60f1f45d6b0dd2b8a9a6f7f0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ea2b698-89", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:41.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2188MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:03:41.079 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:41.104 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-8382fcdb-2eea-4b75-872f-1f38e68028d5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-317552ba-8b42-4d4f-a84c-6bea20b13adb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:41.235 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.235 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.235 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.235 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.249 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.253 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f817e731-74c6-4c6f-8109-5984539d364a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.253 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4a2b3b75-4c02-4601-b40f-1e37d8b022d7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.253 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 317552ba-8b42-4d4f-a84c-6bea20b13adb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:03:41.253 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 15 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:03:41.253 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=15872MB phys_disk=79GB used_disk=15GB total_vcpus=6 used_vcpus=15 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:03:41.288 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:41.288 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:41.504 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:41.516 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:41.516 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:03:41.516 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:42.004 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '1579F53B-32C1-4DB7-9E89-5E405A34F6D1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14947), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:42.130 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:42.130 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:42.130 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:42.130 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:42.147 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1579F53B-32C1-4DB7-9E89-5E405A34F6D1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110935), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:42.147 4076 3051093171896 MainThread DEBUG os_win._utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.577946662902832 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:42.157 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-9dfa6208-783e-483f-80c4-7c506f7d162b - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:42.157 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-9dfa6208-783e-483f-80c4-7c506f7d162b - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:42.157 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:42.172 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:42.204 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-9dfa6208-783e-483f-80c4-7c506f7d162b - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:42.233 4076 3051093168552 MainThread INFO nova.compute.manager [req-9dfa6208-783e-483f-80c4-7c506f7d162b - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:42.308 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:42.308 4076 3051093170664 MainThread DEBUG nova.virt.hardware [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:42.308 4076 3051093170664 MainThread INFO nova.compute.claims [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Claim successful on node n-h1-808474-1 2021-09-10 17:03:42.443 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.hostops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:42.461 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.hostops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:42.673 4076 3051093170664 MainThread DEBUG nova.compute.provider_tree [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:42.692 4076 3051093170664 MainThread DEBUG nova.scheduler.client.report [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:42.692 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:42.692 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:42.782 4076 3051093170664 MainThread DEBUG nova.compute.utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:42.782 4076 3051093168552 GreenThread-79 DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:42.782 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:42.797 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:42.985 4076 3051093168552 GreenThread-79 DEBUG nova.policy [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:42.985 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:43.005 4076 3051093170664 MainThread INFO nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Spawning new instance 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG nova.block_device [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG os_win.utils.pathutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-0000005c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:43.018 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000005c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:43.052 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Resizing VHD C:\openstack\instances\instance-0000005c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:43.178 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '932B55CA-1E34-44CB-84A8-21A419F891B6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15436), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:43.318 4076 3051055217480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:43.318 4076 3051055217480 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:43.318 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:43.318 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:43.318 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '932B55CA-1E34-44CB-84A8-21A419F891B6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110017), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:43.318 4076 3051093171896 MainThread DEBUG os_win._utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7492363452911377 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:43.334 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-d0db545b-ef81-4096-a15a-5423f5a11eed - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:43.334 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [req-d0db545b-ef81-4096-a15a-5423f5a11eed - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:43.384 4076 3051055217480 MainThread DEBUG nova.compute.manager [req-d0db545b-ef81-4096-a15a-5423f5a11eed - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:43.412 4076 3051055217480 MainThread INFO nova.compute.manager [req-d0db545b-ef81-4096-a15a-5423f5a11eed - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:43.499 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:03:43.532 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:43.564 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:43.720 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:43.735 4076 3051093170312 MainThread DEBUG nova.virt.hardware [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:43.735 4076 3051093170312 MainThread INFO nova.compute.claims [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Claim successful on node n-h1-808474-1 2021-09-10 17:03:43.865 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:43.877 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:43.979 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Successfully created port: f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:44.095 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:44.127 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:44.127 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:44.127 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:44.216 4076 3051093170312 MainThread DEBUG nova.compute.utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:44.216 4076 3051055217480 GreenThread-80 DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:44.233 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:44.242 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:44.345 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '0516D47B-6922-41C5-926E-564827638466', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15586), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:44.394 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:44.394 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Spawning new instance 2021-09-10 17:03:44.416 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:44.416 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:44.416 4076 3051093170312 MainThread DEBUG nova.block_device [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:44.416 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:44.416 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:44.427 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Creating directory: C:\openstack\instances\instance-0000005d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:44.427 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000005d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:44.454 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Resizing VHD C:\openstack\instances\instance-0000005d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:44.501 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:44.501 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:44.501 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:44.501 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:44.501 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0516D47B-6922-41C5-926E-564827638466', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114677), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:44.501 4076 3051093171896 MainThread DEBUG os_win._utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.931942939758301 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:44.525 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-4af92409-94a5-443c-a354-35b0713a30fa - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:44.525 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-4af92409-94a5-443c-a354-35b0713a30fa - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:44.525 4076 3051055217480 GreenThread-80 DEBUG nova.policy [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '206940e6bc1a478bb7f92f7778bec0e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5283391577e14a77bbd03e5dcf9f9e5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:44.568 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-4af92409-94a5-443c-a354-35b0713a30fa - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:44.599 4076 3051055214664 MainThread INFO nova.compute.manager [req-4af92409-94a5-443c-a354-35b0713a30fa - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:45.184 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Successfully updated port: f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:45.188 4076 3051093168552 GreenThread-79 DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-442531a0-4fb7-4282-a436-20dfe4a05c72" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:45.208 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:45.240 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Received event network-changed-f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:45.240 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Refreshing instance network info cache due to event network-changed-f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:45.333 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:45.537 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '9D0CF0B3-868A-41B5-8689-54F224FEDD67', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15566), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:45.678 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:45.678 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: f817e731-74c6-4c6f-8109-5984539d364a] VM Stopped (Lifecycle Event) 2021-09-10 17:03:45.678 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:45.692 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:45.692 4076 3051093171896 MainThread ERROR os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000057\' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE8010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000057\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)instance-00000057F01F3E05-29DC-41A1-92E2-FBFCA65ECE80%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9D0CF0B3-868A-41B5-8689-54F224FEDD67', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111274), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\nNot enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)\n\n'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80)", 'ErrorSummaryDescription': "'instance-00000057' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000057.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:45.692 4076 3051093171896 MainThread ERROR nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Failed to change vm state of instance-00000057 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. 2021-09-10 17:03:45.692 4076 3051093171896 MainThread INFO nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Got request to destroy instance 2021-09-10 17:03:45.719 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-0f84c620-c203-41f2-9588-ce0036431013 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:45.719 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-0f84c620-c203-41f2-9588-ce0036431013 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:45.770 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-0f84c620-c203-41f2-9588-ce0036431013 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:45.770 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Successfully created port: 3a943e3a-f01b-490e-be8d-9d68011cbb69 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:45.804 4076 3051055214664 MainThread INFO nova.compute.manager [req-0f84c620-c203-41f2-9588-ce0036431013 - - - - -] [instance: f817e731-74c6-4c6f-8109-5984539d364a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:45.933 4076 3051093168552 GreenThread-79 DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Updating instance_info_cache with network_info: [{"id": "f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e", "address": "fa:16:3e:1a:c3:56", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf6affd4a-b9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:45.959 4076 3051093168552 GreenThread-79 DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-442531a0-4fb7-4282-a436-20dfe4a05c72" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:45.959 4076 3051093168552 GreenThread-79 DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Instance network_info: |[{"id": "f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e", "address": "fa:16:3e:1a:c3:56", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf6affd4a-b9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:45.959 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Preparing to wait for external event network-vif-plugged-f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:45.959 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:45.959 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:45.959 4076 3051093170664 MainThread DEBUG os_win.utils.compute.vmutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-0000005c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:45.959 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-442531a0-4fb7-4282-a436-20dfe4a05c72" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:45.959 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Refreshing network info cache for port f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:46.697 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5a1f4a8e-3a6c-402d-831f-4a968e3fc6f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Received event network-vif-plugged-9ea2b698-897f-4c4a-80d8-f74194d1fead external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:46.697 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5a1f4a8e-3a6c-402d-831f-4a968e3fc6f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:46.697 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5a1f4a8e-3a6c-402d-831f-4a968e3fc6f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:46.697 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5a1f4a8e-3a6c-402d-831f-4a968e3fc6f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Processing event network-vif-plugged-9ea2b698-897f-4c4a-80d8-f74194d1fead _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:46.738 4076 3051055218008 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Using config drive for instance 2021-09-10 17:03:46.941 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Updated VIF entry in instance network info cache for port f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:46.941 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Updating instance_info_cache with network_info: [{"id": "f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e", "address": "fa:16:3e:1a:c3:56", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf6affd4a-b9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:46.941 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:46.959 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-8741801f-503c-4824-b302-b1d783fcd0a7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-442531a0-4fb7-4282-a436-20dfe4a05c72" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:47.123 4076 3051055218008 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Creating config drive at C:\openstack\instances\instance-0000005a\configdrive.iso 2021-09-10 17:03:47.157 4076 3051055218008 MainThread DEBUG oslo_concurrency.processutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp7n_awd40 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:47.304 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Successfully updated port: 3a943e3a-f01b-490e-be8d-9d68011cbb69 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:03:47.318 4076 3051055217480 GreenThread-80 DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Acquired lock "refresh_cache-d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:47.335 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:03:47.475 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:03:47.475 4076 3051055218008 MainThread DEBUG oslo_concurrency.processutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp7n_awd40" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:47.745 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Received event network-changed-3a943e3a-f01b-490e-be8d-9d68011cbb69 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:47.745 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Refreshing instance network info cache due to event network-changed-3a943e3a-f01b-490e-be8d-9d68011cbb69. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:03:47.801 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:47.801 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:47.801 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:47.816 4076 3051093171896 MainThread DEBUG os_win.utils.compute.vmutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Successfully changed vm state of instance-00000057 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:47.816 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Successfully changed state of VM instance-00000057 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:47.871 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Waiting for WMI job: {'InstanceID': '23218A40-3BEE-4F0F-8320-A4D97A97DA77', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14868), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:47.910 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Setting boot order for instance: instance-0000005a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:47.944 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:47.992 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '05C4EE52-E37C-47F3-B659-8C02061F318F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15887), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:47.992 4076 3051093171896 MainThread DEBUG os_win.utils.jobutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] WMI job succeeded: {'InstanceID': '23218A40-3BEE-4F0F-8320-A4D97A97DA77', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30390), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:47.992 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:47.992 4076 3051093171896 MainThread DEBUG os_win.utils.pathutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Removing directory: C:\openstack\instances\instance-00000057 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:47.992 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:48.013 4076 3051093171896 MainThread ERROR nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Traceback (most recent call last): 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(*args, **kwargs) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] admin_password, network_info, block_device_info) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(self, *args, **kwds) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] should_plug_vifs=False) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'req_state': req_state}) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] time.sleep(sleep_time) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return f(*args, **kwargs) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return self._wait_for_job(job_path) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] error_desc=err_desc) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] During handling of the above exception, another exception occurred: 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Traceback (most recent call last): 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] yield resources 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] accel_info=accel_info) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(*args, **kwargs) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] admin_password, network_info, block_device_info) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(self, *args, **kwds) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] should_plug_vifs=False) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'req_state': req_state}) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] time.sleep(sleep_time) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return f(*args, **kwargs) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return self._wait_for_job(job_path) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] error_desc=err_desc) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.013 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.013 4076 3051093171896 MainThread INFO nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Terminating instance 2021-09-10 17:03:48.013 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:48.013 4076 3051093171896 MainThread INFO nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Got request to destroy instance 2021-09-10 17:03:48.032 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:48.032 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:48.032 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "instance-00000057" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:48.038 4076 3051093171896 MainThread INFO nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:48.038 4076 3051093171896 MainThread DEBUG nova.compute.claims [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:48.038 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:48.038 4076 3051055217480 GreenThread-80 DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Updating instance_info_cache with network_info: [{"id": "3a943e3a-f01b-490e-be8d-9d68011cbb69", "address": "fa:16:3e:ad:31:b7", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3a943e3a-f0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:48.087 4076 3051055217480 GreenThread-80 DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Releasing lock "refresh_cache-d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:48.087 4076 3051055217480 GreenThread-80 DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Instance network_info: |[{"id": "3a943e3a-f01b-490e-be8d-9d68011cbb69", "address": "fa:16:3e:ad:31:b7", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3a943e3a-f0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:03:48.087 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Preparing to wait for external event network-vif-plugged-3a943e3a-f01b-490e-be8d-9d68011cbb69 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:03:48.087 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:48.087 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:48.087 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Creating VM instance-0000005d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:03:48.087 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:03:48.087 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Refreshing network info cache for port 3a943e3a-f01b-490e-be8d-9d68011cbb69 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:03:48.132 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:48.132 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:48.132 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:48.144 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:48.144 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '05C4EE52-E37C-47F3-B659-8C02061F318F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115101), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:48.144 4076 3051055218008 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.2003345489501953 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:48.157 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-ff489cf9-0c10-48c6-8839-fd8a95041a00 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:48.157 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-ff489cf9-0c10-48c6-8839-fd8a95041a00 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:48.185 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.hostops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:48.189 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.hostops [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:48.224 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-ff489cf9-0c10-48c6-8839-fd8a95041a00 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:48.286 4076 3051055216776 MainThread INFO nova.compute.manager [req-ff489cf9-0c10-48c6-8839-fd8a95041a00 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:48.416 4076 3051093171896 MainThread DEBUG nova.compute.provider_tree [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.scheduler.client.report [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.516s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:48.551 4076 3051093171896 MainThread ERROR nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Traceback (most recent call last): 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(*args, **kwargs) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] admin_password, network_info, block_device_info) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(self, *args, **kwds) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] should_plug_vifs=False) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'req_state': req_state}) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] time.sleep(sleep_time) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return f(*args, **kwargs) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return self._wait_for_job(job_path) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] error_desc=err_desc) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] During handling of the above exception, another exception occurred: 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Traceback (most recent call last): 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] accel_info=accel_info) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(*args, **kwargs) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] admin_password, network_info, block_device_info) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return function(self, *args, **kwds) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] should_plug_vifs=False) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'req_state': req_state}) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] time.sleep(sleep_time) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self.force_reraise() 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] raise self.value 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return f(*args, **kwargs) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] return self._wait_for_job(job_path) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] error_desc=err_desc) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. 2021-09-10 17:03:48.551 4076 3051093171896 MainThread TRACE nova.compute.manager [instance: f817e731-74c6-4c6f-8109-5984539d364a] 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.compute.utils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Build of instance f817e731-74c6-4c6f-8109-5984539d364a was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000057' failed to start. Not enough memory in the system to start the virtual machine instance-00000057. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000057' failed to start. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Not enough memory in the system to start the virtual machine instance-00000057 with ram size 1024 megabytes. (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) 'instance-00000057' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F01F3E05-29DC-41A1-92E2-FBFCA65ECE80) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:48.551 4076 3051093171896 MainThread DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:49.083 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:03:49.158 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': 'DAF9353F-5FB0-41C4-A840-D8008FF134C1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15153), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:49.318 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:49.318 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:49.318 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:49.318 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:49.318 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DAF9353F-5FB0-41C4-A840-D8008FF134C1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118244), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:49.318 4076 3051055218008 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3737711906433105 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:49.352 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-5eb7f017-0970-4064-9f9d-bb951ef80bb9 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:49.352 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-5eb7f017-0970-4064-9f9d-bb951ef80bb9 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:49.376 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-5eb7f017-0970-4064-9f9d-bb951ef80bb9 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:49.438 4076 3051113294664 MainThread INFO nova.compute.manager [req-5eb7f017-0970-4064-9f9d-bb951ef80bb9 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:49.444 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Updated VIF entry in instance network info cache for port 3a943e3a-f01b-490e-be8d-9d68011cbb69. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:03:49.444 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Updating instance_info_cache with network_info: [{"id": "3a943e3a-f01b-490e-be8d-9d68011cbb69", "address": "fa:16:3e:ad:31:b7", "network": {"id": "e365bc30-8c38-4b70-a29c-e076c845f9de", "bridge": null, "label": "tempest-AttachVolumeTestJSON-1488926721-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5283391577e14a77bbd03e5dcf9f9e5c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3a943e3a-f0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:49.460 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-4e54c33f-c5df-4d9c-b1bc-c2a76b467773 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:03:49.600 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a58467f-ecf6-45ee-8ec1-7945f04d2551 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Received event network-vif-plugged-fc5f4b1e-56fb-453c-b480-f6987341eda9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:49.600 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a58467f-ecf6-45ee-8ec1-7945f04d2551 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:49.600 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-7a58467f-ecf6-45ee-8ec1-7945f04d2551 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:49.600 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-7a58467f-ecf6-45ee-8ec1-7945f04d2551 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Processing event network-vif-plugged-fc5f4b1e-56fb-453c-b480-f6987341eda9 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:49.652 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Using config drive for instance 2021-09-10 17:03:49.959 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Creating config drive at C:\openstack\instances\instance-00000059\configdrive.iso 2021-09-10 17:03:50.007 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000059\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps8gciuqp execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:50.298 4076 3051093171896 MainThread DEBUG nova.network.neutron [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:50.313 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000059\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps8gciuqp" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:50.347 4076 3051093171896 MainThread INFO nova.compute.manager [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] [instance: f817e731-74c6-4c6f-8109-5984539d364a] Took 1.80 seconds to deallocate network for instance. 2021-09-10 17:03:50.347 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '3FAF0BA5-F3A6-4635-8994-C010CAEC5D99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=33598), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:50.477 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:50.477 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:50.477 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:50.491 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:50.491 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3FAF0BA5-F3A6-4635-8994-C010CAEC5D99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117330), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:50.491 4076 3051055218008 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.547293186187744 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:50.517 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-2f904396-af44-43aa-8910-99f25b40990c - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:50.517 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-2f904396-af44-43aa-8910-99f25b40990c - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:50.569 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-2f904396-af44-43aa-8910-99f25b40990c - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:50.613 4076 3051093171896 MainThread INFO nova.scheduler.client.report [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Deleted allocations for instance f817e731-74c6-4c6f-8109-5984539d364a 2021-09-10 17:03:50.631 4076 3051093168552 MainThread INFO nova.compute.manager [req-2f904396-af44-43aa-8910-99f25b40990c - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:50.682 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-93177b03-ec0a-493f-bf8c-a47bdb5b1966 7f13c7f24aed435daa5c4c3caf5d68c9 3f7a5fed42ca440bb659840e05ce1f95 - default default] Lock "f817e731-74c6-4c6f-8109-5984539d364a" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 23.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:50.768 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Setting boot order for instance: instance-00000059: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:50.824 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:50.849 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '6B09275A-E1C2-4D23-A16F-D93626152DF8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15376), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:50.991 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6B09275A-E1C2-4D23-A16F-D93626152DF8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120281), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:50.991 4076 3051055217128 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16728734970092773 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:51.001 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:51.001 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:51.001 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:51.001 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:51.038 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-0824dbdd-4866-428a-9728-5eeb65a0003c - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:51.038 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-0824dbdd-4866-428a-9728-5eeb65a0003c - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:51.085 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-0824dbdd-4866-428a-9728-5eeb65a0003c - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:51.111 4076 3051093171896 MainThread INFO nova.compute.manager [req-0824dbdd-4866-428a-9728-5eeb65a0003c - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:51.506 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '264F1A77-E6F8-493A-BFCF-928450C49011', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14551), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:51.643 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:51.643 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:51.658 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:51.658 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:51.658 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '264F1A77-E6F8-493A-BFCF-928450C49011', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112847), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:51.658 4076 3051055218008 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.714498281478882 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:51.692 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-f22610d6-acc4-41e6-9409-d1b03b266539 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:51.692 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-f22610d6-acc4-41e6-9409-d1b03b266539 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:51.719 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-f22610d6-acc4-41e6-9409-d1b03b266539 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:51.760 4076 3051093171896 MainThread INFO nova.compute.manager [req-f22610d6-acc4-41e6-9409-d1b03b266539 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:52.017 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': 'EA6EAE65-D865-4321-B181-1A258EE9DE92', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15633), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:52.148 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:52.148 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:52.148 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:52.148 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:52.173 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EA6EAE65-D865-4321-B181-1A258EE9DE92', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112733), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:52.173 4076 3051055217128 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3485288619995117 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:52.173 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-6b14082a-62f5-4d77-b35b-6d4680f8ab20 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:52.173 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-6b14082a-62f5-4d77-b35b-6d4680f8ab20 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:52.222 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-6b14082a-62f5-4d77-b35b-6d4680f8ab20 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:52.277 4076 3051093171896 MainThread INFO nova.compute.manager [req-6b14082a-62f5-4d77-b35b-6d4680f8ab20 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:52.695 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '3115959E-9B23-48B1-BD1A-A62BD57C4C16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14368), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:52.832 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:52.832 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:52.832 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:52.832 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:52.832 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3115959E-9B23-48B1-BD1A-A62BD57C4C16', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114322), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:52.832 4076 3051055218008 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.888279676437378 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:52.863 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-c615e547-1bce-4096-ba3e-5d44cb816e9a - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:52.863 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-c615e547-1bce-4096-ba3e-5d44cb816e9a - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:52.916 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-c615e547-1bce-4096-ba3e-5d44cb816e9a - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:52.929 4076 3051093171896 MainThread INFO nova.compute.manager [req-c615e547-1bce-4096-ba3e-5d44cb816e9a - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:53.207 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '245C6641-FCD4-4B43-B933-BE8D8FF3591C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15461), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:53.350 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:53.350 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:53.350 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:53.350 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:53.350 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '245C6641-FCD4-4B43-B933-BE8D8FF3591C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113329), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:53.350 4076 3051055217128 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.526047468185425 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:53.375 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-8ee2260c-3aa7-48f9-a38f-f0308a367a4b - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:53.375 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-8ee2260c-3aa7-48f9-a38f-f0308a367a4b - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:53.412 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-8ee2260c-3aa7-48f9-a38f-f0308a367a4b - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:53.458 4076 3051093171896 MainThread INFO nova.compute.manager [req-8ee2260c-3aa7-48f9-a38f-f0308a367a4b - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:53.877 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '56798BCB-B027-483D-9E45-2582FEA5AFDF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15546), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:54.008 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:54.008 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] VM Stopped (Lifecycle Event) 2021-09-10 17:03:54.008 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:54.008 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:54.020 4076 3051055218008 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005a\' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)instance-0000005a27A2C376-FCC8-43BA-B972-CA350506DC6F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '56798BCB-B027-483D-9E45-2582FEA5AFDF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113894), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\nNot enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)\n\n'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F)", 'ErrorSummaryDescription': "'instance-0000005a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:54.020 4076 3051055218008 MainThread ERROR nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Failed to change vm state of instance-0000005a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. 2021-09-10 17:03:54.020 4076 3051055218008 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Got request to destroy instance 2021-09-10 17:03:54.055 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-8bcf163d-a9d0-4d65-9668-9c6a77238401 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:54.055 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-8bcf163d-a9d0-4d65-9668-9c6a77238401 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:54.104 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-8bcf163d-a9d0-4d65-9668-9c6a77238401 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:54.148 4076 3051093171896 MainThread INFO nova.compute.manager [req-8bcf163d-a9d0-4d65-9668-9c6a77238401 - - - - -] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:54.377 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '40E5BB29-10F9-4A90-97A7-47D76FD8ACE6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15435), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:54.522 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:54.522 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:54.522 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:54.522 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:54.522 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '40E5BB29-10F9-4A90-97A7-47D76FD8ACE6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115136), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 54), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 54), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:54.522 4076 3051055217128 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.697737455368042 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:54.547 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-cbc3ed64-eb05-4b18-a5f0-6b92b70d27c0 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:54.547 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-cbc3ed64-eb05-4b18-a5f0-6b92b70d27c0 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:54.585 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-cbc3ed64-eb05-4b18-a5f0-6b92b70d27c0 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:54.616 4076 3051093171896 MainThread INFO nova.compute.manager [req-cbc3ed64-eb05-4b18-a5f0-6b92b70d27c0 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:54.813 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-7155a60e-4e2d-4565-a1eb-99a8692ca8f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Received event network-vif-plugged-f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:54.813 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-7155a60e-4e2d-4565-a1eb-99a8692ca8f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:54.813 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [req-7155a60e-4e2d-4565-a1eb-99a8692ca8f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:54.813 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-7155a60e-4e2d-4565-a1eb-99a8692ca8f5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Processing event network-vif-plugged-f6affd4a-b9f7-4f45-aeab-7e559fb7ce4e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:54.879 4076 3051093170664 MainThread INFO nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Using config drive for instance 2021-09-10 17:03:55.037 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "45310242-58a0-47b9-9655-a45945084513" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:55.067 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:03:55.180 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:55.194 4076 3051093170136 MainThread DEBUG nova.virt.hardware [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:03:55.194 4076 3051093170136 MainThread INFO nova.compute.claims [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Claim successful on node n-h1-808474-1 2021-09-10 17:03:55.194 4076 3051093170664 MainThread INFO nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Creating config drive at C:\openstack\instances\instance-0000005c\configdrive.iso 2021-09-10 17:03:55.239 4076 3051093170664 MainThread DEBUG oslo_concurrency.processutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmptaldd_qn execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:55.439 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:55.456 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:55.553 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': 'F7B15C2D-F27A-46AE-9D15-611C5A7C9498', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20433), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:55.553 4076 3051093170664 MainThread DEBUG oslo_concurrency.processutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmptaldd_qn" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:55.704 4076 3051093170136 MainThread DEBUG nova.compute.provider_tree [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:55.704 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:55.704 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:55.704 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:55.704 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:55.725 4076 3051093170136 MainThread DEBUG nova.scheduler.client.report [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:55.725 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:55.725 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:03:55.725 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-4ba878c6-2178-499f-ad65-b6e38b252c62 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:55.725 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-4ba878c6-2178-499f-ad65-b6e38b252c62 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:55.783 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-4ba878c6-2178-499f-ad65-b6e38b252c62 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:55.802 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F7B15C2D-F27A-46AE-9D15-611C5A7C9498', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=131684), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:55.802 4076 3051055217128 MainThread DEBUG os_win._utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.977672100067139 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:55.813 4076 3051093170136 MainThread DEBUG nova.compute.utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:03:55.813 4076 3051093171896 GreenThread-81 DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:03:55.813 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:03:55.813 4076 3051093168552 MainThread INFO nova.compute.manager [req-4ba878c6-2178-499f-ad65-b6e38b252c62 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:55.842 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:03:55.980 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:03:55.980 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Spawning new instance 2021-09-10 17:03:56.024 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.024 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.024 4076 3051093170136 MainThread DEBUG nova.block_device [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:03:56.032 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.032 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.032 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-0000005c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:56.032 4076 3051093170136 MainThread DEBUG os_win.utils.pathutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-0000005f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:03:56.032 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000005f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:03:56.050 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Resizing VHD C:\openstack\instances\instance-0000005f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:03:56.082 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:56.113 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '8587CF8E-CA8D-4073-86B0-31050326CB9E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18539), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:56.134 4076 3051093171896 GreenThread-81 DEBUG nova.policy [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:03:56.189 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:56.189 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.189 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.189 4076 3051055218008 MainThread DEBUG os_win.utils.compute.vmutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Successfully changed vm state of instance-0000005a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:56.189 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Successfully changed state of VM instance-0000005a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:56.238 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.238 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.238 4076 3051113294664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:56.270 4076 3051113294664 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:03:56.270 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': 'F50E818F-539D-4722-A139-CB4B6DAB5621', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15527), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:56.270 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8587CF8E-CA8D-4073-86B0-31050326CB9E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115235), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:56.270 4076 3051093170664 MainThread DEBUG os_win._utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1885974407196045 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:56.288 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9fe24b6a-e6b3-4570-9a68-94eb2fd2fcd4 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:56.288 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-9fe24b6a-e6b3-4570-9a68-94eb2fd2fcd4 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:56.344 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-9fe24b6a-e6b3-4570-9a68-94eb2fd2fcd4 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:56.350 4076 3051113294664 MainThread INFO nova.compute.manager [req-9fe24b6a-e6b3-4570-9a68-94eb2fd2fcd4 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job succeeded: {'InstanceID': 'F50E818F-539D-4722-A139-CB4B6DAB5621', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29610), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:56.384 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.384 4076 3051055218008 MainThread DEBUG os_win.utils.pathutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Removing directory: C:\openstack\instances\instance-0000005a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:56.384 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.384 4076 3051055218008 MainThread ERROR nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Traceback (most recent call last): 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(*args, **kwargs) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] admin_password, network_info, block_device_info) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(self, *args, **kwds) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] should_plug_vifs=False) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'req_state': req_state}) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] time.sleep(sleep_time) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return f(*args, **kwargs) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return self._wait_for_job(job_path) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] error_desc=err_desc) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During handling of the above exception, another exception occurred: 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Traceback (most recent call last): 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] yield resources 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] accel_info=accel_info) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(*args, **kwargs) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] admin_password, network_info, block_device_info) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(self, *args, **kwds) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] should_plug_vifs=False) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'req_state': req_state}) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] time.sleep(sleep_time) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return f(*args, **kwargs) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return self._wait_for_job(job_path) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] error_desc=err_desc) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.384 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.384 4076 3051055218008 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Terminating instance 2021-09-10 17:03:56.384 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:56.384 4076 3051055218008 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Got request to destroy instance 2021-09-10 17:03:56.406 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:56.406 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.406 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-0000005a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.421 4076 3051055218008 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:56.421 4076 3051055218008 MainThread DEBUG nova.compute.claims [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:56.421 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.569 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:56.587 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:56.806 4076 3051055218008 MainThread DEBUG nova.compute.provider_tree [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.830 4076 3051055218008 MainThread ERROR nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Traceback (most recent call last): 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(*args, **kwargs) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] admin_password, network_info, block_device_info) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(self, *args, **kwds) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] should_plug_vifs=False) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'req_state': req_state}) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] time.sleep(sleep_time) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return f(*args, **kwargs) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return self._wait_for_job(job_path) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] error_desc=err_desc) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] During handling of the above exception, another exception occurred: 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Traceback (most recent call last): 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] accel_info=accel_info) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(*args, **kwargs) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] admin_password, network_info, block_device_info) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return function(self, *args, **kwds) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] should_plug_vifs=False) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'req_state': req_state}) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] time.sleep(sleep_time) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self.force_reraise() 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] raise self.value 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return f(*args, **kwargs) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] return self._wait_for_job(job_path) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] error_desc=err_desc) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. 2021-09-10 17:03:56.830 4076 3051055218008 MainThread TRACE nova.compute.manager [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.compute.utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Build of instance 317552ba-8b42-4d4f-a84c-6bea20b13adb was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000005a' failed to start. Not enough memory in the system to start the virtual machine instance-0000005a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005a' failed to start. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Not enough memory in the system to start the virtual machine instance-0000005a with ram size 1024 megabytes. (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) 'instance-0000005a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 27A2C376-FCC8-43BA-B972-CA350506DC6F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:56.830 4076 3051055218008 MainThread DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:56.830 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': 'ACE3C5AF-8AEE-41BE-A909-3EB5B9A472B0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17342), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:56.876 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-e1649df1-75de-41f1-a00c-ba632bd7b6e7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Received event network-vif-plugged-3a943e3a-f01b-490e-be8d-9d68011cbb69 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:03:56.876 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-e1649df1-75de-41f1-a00c-ba632bd7b6e7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.876 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-e1649df1-75de-41f1-a00c-ba632bd7b6e7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.876 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-e1649df1-75de-41f1-a00c-ba632bd7b6e7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Processing event network-vif-plugged-3a943e3a-f01b-490e-be8d-9d68011cbb69 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:03:56.894 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Using config drive for instance 2021-09-10 17:03:56.966 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:56.976 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] VM Stopped (Lifecycle Event) 2021-09-10 17:03:56.976 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:56.976 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:56.976 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job failed: {'RawErrors': (('124\'instance-00000059\' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000059\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)instance-000000591A7F7656-EFBE-45BE-B0E6-AE90A4868A4E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'ACE3C5AF-8AEE-41BE-A909-3EB5B9A472B0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113669), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\nNot enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)\n\n'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E)", 'ErrorSummaryDescription': "'instance-00000059' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000059.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:56.976 4076 3051055217128 MainThread ERROR nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Failed to change vm state of instance-00000059 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. 2021-09-10 17:03:56.996 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Got request to destroy instance 2021-09-10 17:03:56.996 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-f2326c5c-525f-417a-a2d1-0bc073aa02c2 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:56.996 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-f2326c5c-525f-417a-a2d1-0bc073aa02c2 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:57.038 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-f2326c5c-525f-417a-a2d1-0bc073aa02c2 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:57.065 4076 3051113162008 MainThread INFO nova.compute.manager [req-f2326c5c-525f-417a-a2d1-0bc073aa02c2 - - - - -] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:57.287 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '4B6DF861-1DC5-4039-A166-E71D62BA2872', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14877), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:57.367 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Creating config drive at C:\openstack\instances\instance-0000005d\configdrive.iso 2021-09-10 17:03:57.412 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp38tpo8dq execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:03:57.548 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Successfully created port: bd224892-0bf2-4c64-a8c7-c69883004896 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:03:57.584 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:57.584 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:57.584 4076 3051113161480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:57.584 4076 3051113161480 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:03:57.595 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4B6DF861-1DC5-4039-A166-E71D62BA2872', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111725), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:57.595 4076 3051093170664 MainThread DEBUG os_win._utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.51338529586792 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:57.613 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-9a835d69-eefd-4b8d-9468-6fb5d562300f - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:57.613 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-9a835d69-eefd-4b8d-9468-6fb5d562300f - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:57.661 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-9a835d69-eefd-4b8d-9468-6fb5d562300f - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:57.679 4076 3051113161480 MainThread INFO nova.compute.manager [req-9a835d69-eefd-4b8d-9468-6fb5d562300f - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:57.752 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp38tpo8dq" returned: 0 in 0.344s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:03:58.194 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Setting boot order for instance: instance-0000005d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:03:58.250 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:03:58.282 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': 'CB10E172-29E3-4804-8F38-582A8913C937', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16641), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:58.392 4076 3051055218008 MainThread DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:03:58.424 4076 3051113163240 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:58.424 4076 3051113163240 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:58.424 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:58.424 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:58.424 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CB10E172-29E3-4804-8F38-582A8913C937', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118359), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:58.424 4076 3051093170312 MainThread DEBUG os_win._utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1737813949584961 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:58.457 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-daee1837-aceb-4a77-a1e2-ba9412496e9e - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:58.457 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-daee1837-aceb-4a77-a1e2-ba9412496e9e - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:58.457 4076 3051055218008 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 317552ba-8b42-4d4f-a84c-6bea20b13adb] Took 1.62 seconds to deallocate network for instance. 2021-09-10 17:03:58.488 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-daee1837-aceb-4a77-a1e2-ba9412496e9e - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:58.538 4076 3051113163240 MainThread INFO nova.compute.manager [req-daee1837-aceb-4a77-a1e2-ba9412496e9e - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:58.628 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'F9F9FFDF-D8C7-4279-93D4-598B277D3DF6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14529), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:58.753 4076 3051055218008 MainThread INFO nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Deleted allocations for instance 317552ba-8b42-4d4f-a84c-6bea20b13adb 2021-09-10 17:03:58.774 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:58.774 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:58.774 4076 3051113163240 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:58.774 4076 3051113163240 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:03:58.788 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9F9FFDF-D8C7-4279-93D4-598B277D3DF6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115349), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:58.788 4076 3051093170664 MainThread DEBUG os_win._utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.7060365676879883 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:58.802 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "317552ba-8b42-4d4f-a84c-6bea20b13adb" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.969s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:58.866 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-75d7b67c-bbcd-4c6a-b37b-5617643b1372 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:58.866 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-75d7b67c-bbcd-4c6a-b37b-5617643b1372 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:58.916 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-75d7b67c-bbcd-4c6a-b37b-5617643b1372 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:58.946 4076 3051113163240 MainThread INFO nova.compute.manager [req-75d7b67c-bbcd-4c6a-b37b-5617643b1372 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:59.116 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:03:59.116 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.116 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.125 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Successfully changed vm state of instance-00000059 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:03:59.125 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Successfully changed state of VM instance-00000059 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:03:59.159 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Waiting for WMI job: {'InstanceID': '6AD695FD-9E71-4D8A-9ADF-B9E163BF9389', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14150), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:59.275 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] WMI job succeeded: {'InstanceID': '6AD695FD-9E71-4D8A-9ADF-B9E163BF9389', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30313), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:03:59.275 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.275 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Removing directory: C:\openstack\instances\instance-00000059 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:03:59.275 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.275 4076 3051055217128 MainThread ERROR nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Traceback (most recent call last): 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(*args, **kwargs) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] admin_password, network_info, block_device_info) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(self, *args, **kwds) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] should_plug_vifs=False) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'req_state': req_state}) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] time.sleep(sleep_time) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return f(*args, **kwargs) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return self._wait_for_job(job_path) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] error_desc=err_desc) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During handling of the above exception, another exception occurred: 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Traceback (most recent call last): 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] yield resources 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] accel_info=accel_info) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(*args, **kwargs) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] admin_password, network_info, block_device_info) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(self, *args, **kwds) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] should_plug_vifs=False) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'req_state': req_state}) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] time.sleep(sleep_time) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return f(*args, **kwargs) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return self._wait_for_job(job_path) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] error_desc=err_desc) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.275 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.297 4076 3051055217128 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Terminating instance 2021-09-10 17:03:59.297 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:03:59.297 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Got request to destroy instance 2021-09-10 17:03:59.317 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:03:59.317 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.317 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "instance-00000059" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.317 4076 3051055217128 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:03:59.317 4076 3051055217128 MainThread DEBUG nova.compute.claims [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:03:59.317 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.444 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '330DEDC5-80AE-435E-B268-AFE961C62B99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14235), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:59.487 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:03:59.487 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:03:59.567 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.567 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.567 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:59.567 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:03:59.591 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '330DEDC5-80AE-435E-B268-AFE961C62B99', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111394), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:59.591 4076 3051093170312 MainThread DEBUG os_win._utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3411643505096436 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:59.616 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-a6794e49-d4b3-4955-9f2a-1609e519361c - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:59.616 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-a6794e49-d4b3-4955-9f2a-1609e519361c - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:03:59.645 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-a6794e49-d4b3-4955-9f2a-1609e519361c - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:03:59.703 4076 3051055218008 MainThread INFO nova.compute.manager [req-a6794e49-d4b3-4955-9f2a-1609e519361c - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:03:59.722 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:03:59.722 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.722 4076 3051055217128 MainThread ERROR nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Traceback (most recent call last): 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(*args, **kwargs) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] admin_password, network_info, block_device_info) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(self, *args, **kwds) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] should_plug_vifs=False) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'req_state': req_state}) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] time.sleep(sleep_time) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return f(*args, **kwargs) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return self._wait_for_job(job_path) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] error_desc=err_desc) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] During handling of the above exception, another exception occurred: 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Traceback (most recent call last): 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] accel_info=accel_info) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(*args, **kwargs) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] admin_password, network_info, block_device_info) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return function(self, *args, **kwds) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] should_plug_vifs=False) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'req_state': req_state}) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] time.sleep(sleep_time) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self.force_reraise() 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] raise self.value 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return f(*args, **kwargs) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] return self._wait_for_job(job_path) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] error_desc=err_desc) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. 2021-09-10 17:03:59.722 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] 2021-09-10 17:03:59.722 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:03:59.750 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Build of instance 4a2b3b75-4c02-4601-b40f-1e37d8b022d7 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000059' failed to start. Not enough memory in the system to start the virtual machine instance-00000059. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000059' failed to start. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Not enough memory in the system to start the virtual machine instance-00000059 with ram size 1024 megabytes. (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) 'instance-00000059' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1A7F7656-EFBE-45BE-B0E6-AE90A4868A4E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:03:59.750 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:03:59.750 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:03:59.750 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:03:59.750 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:03:59.816 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'EF4E52D9-415E-4726-B560-6763D0577637', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14671), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:03:59.946 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:03:59.946 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:03:59.946 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:03:59.946 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:03:59.958 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EF4E52D9-415E-4726-B560-6763D0577637', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111712), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 3, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 3, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:03:59.976 4076 3051093170664 MainThread DEBUG os_win._utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8944995403289795 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:03:59.976 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-894c5e28-16b2-40ad-9504-91dc2b004235 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:03:59.976 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-894c5e28-16b2-40ad-9504-91dc2b004235 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:00.019 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-894c5e28-16b2-40ad-9504-91dc2b004235 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:00.047 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Successfully updated port: bd224892-0bf2-4c64-a8c7-c69883004896 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:04:00.054 4076 3051113161304 MainThread INFO nova.compute.manager [req-894c5e28-16b2-40ad-9504-91dc2b004235 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:00.054 4076 3051093171896 GreenThread-81 DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-45310242-58a0-47b9-9655-a45945084513" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:00.054 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:04:00.160 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:04:00.616 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '4C48BC60-FFB1-4343-990C-F8ACFAD19879', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15530), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:00.729 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:00.754 4076 3051055217128 MainThread INFO nova.compute.manager [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] [instance: 4a2b3b75-4c02-4601-b40f-1e37d8b022d7] Took 1.00 seconds to deallocate network for instance. 2021-09-10 17:04:00.754 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:00.754 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:00.754 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:00.754 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:04:00.754 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4C48BC60-FFB1-4343-990C-F8ACFAD19879', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112334), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:00.778 4076 3051093170312 MainThread DEBUG os_win._utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.528001308441162 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:00.821 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-016112ff-a8f9-4174-a352-9ebce003ac4a - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:00.821 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-016112ff-a8f9-4174-a352-9ebce003ac4a - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:00.875 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-016112ff-a8f9-4174-a352-9ebce003ac4a - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:00.909 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Received event network-changed-bd224892-0bf2-4c64-a8c7-c69883004896 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:00.909 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Refreshing instance network info cache due to event network-changed-bd224892-0bf2-4c64-a8c7-c69883004896. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:04:00.956 4076 3051113163768 MainThread INFO nova.compute.manager [req-016112ff-a8f9-4174-a352-9ebce003ac4a - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:01.007 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '7827F8BE-37EB-471D-9EAF-605331B20469', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16408), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:01.084 4076 3051055217128 MainThread INFO nova.scheduler.client.report [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Deleted allocations for instance 4a2b3b75-4c02-4601-b40f-1e37d8b022d7 2021-09-10 17:04:01.131 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-622b339b-8955-4c6c-b50a-facf7d4bb938 f602a288fe3b4ea0bf8af5b656e8681a 7055b5c60f1f45d6b0dd2b8a9a6f7f0a - default default] Lock "4a2b3b75-4c02-4601-b40f-1e37d8b022d7" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:01.159 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:01.159 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:01.159 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:01.159 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:04:01.159 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7827F8BE-37EB-471D-9EAF-605331B20469', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=125824), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:01.159 4076 3051093170664 MainThread DEBUG os_win._utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.076735734939575 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:01.188 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-18c624d7-c8d6-4b4b-93a3-4b72744c8447 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:01.188 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-18c624d7-c8d6-4b4b-93a3-4b72744c8447 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:01.234 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-18c624d7-c8d6-4b4b-93a3-4b72744c8447 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:01.278 4076 3051055217128 MainThread INFO nova.compute.manager [req-18c624d7-c8d6-4b4b-93a3-4b72744c8447 - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:01.503 4076 3051093171896 GreenThread-81 DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Updating instance_info_cache with network_info: [{"id": "bd224892-0bf2-4c64-a8c7-c69883004896", "address": "fa:16:3e:85:b4:22", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbd224892-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:01.549 4076 3051093171896 GreenThread-81 DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-45310242-58a0-47b9-9655-a45945084513" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:01.549 4076 3051093171896 GreenThread-81 DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Instance network_info: |[{"id": "bd224892-0bf2-4c64-a8c7-c69883004896", "address": "fa:16:3e:85:b4:22", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbd224892-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:04:01.549 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-45310242-58a0-47b9-9655-a45945084513" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:01.549 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Refreshing network info cache for port bd224892-0bf2-4c64-a8c7-c69883004896 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:04:01.549 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Preparing to wait for external event network-vif-plugged-bd224892-0bf2-4c64-a8c7-c69883004896 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:04:01.549 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "45310242-58a0-47b9-9655-a45945084513-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:01.549 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "45310242-58a0-47b9-9655-a45945084513-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:01.549 4076 3051093170136 MainThread DEBUG os_win.utils.compute.vmutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-0000005f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:04:01.816 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '8FFA37D6-FD57-47EC-942F-D1F9C4252F8F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16713), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:01.957 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:01.957 4076 3051093171896 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:01.957 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:01.957 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:04:01.957 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8FFA37D6-FD57-47EC-942F-D1F9C4252F8F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115805), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:01.980 4076 3051093170312 MainThread DEBUG os_win._utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7296948432922363 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:01.986 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-4cf2e5ab-c882-43df-99c6-1a87b30e34d0 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:02.010 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-4cf2e5ab-c882-43df-99c6-1a87b30e34d0 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:02.050 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-4cf2e5ab-c882-43df-99c6-1a87b30e34d0 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:02.095 4076 3051055217128 MainThread INFO nova.compute.manager [req-4cf2e5ab-c882-43df-99c6-1a87b30e34d0 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:02.179 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'D88E9312-643C-4CF5-AD9A-814075E84E40', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15585), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:02.313 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:02.313 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:02.313 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:02.313 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] VM Stopped (Lifecycle Event) 2021-09-10 17:04:02.313 4076 3051093170664 MainThread ERROR os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005c\' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE0210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)instance-0000005cA7F73905-E9A8-4792-A855-0C27BBC5BE02%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D88E9312-643C-4CF5-AD9A-814075E84E40', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111474), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\nNot enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)\n\n'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02)", 'ErrorSummaryDescription': "'instance-0000005c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:02.313 4076 3051093170664 MainThread ERROR nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-0000005c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. 2021-09-10 17:04:02.313 4076 3051093170664 MainThread INFO nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Got request to destroy instance 2021-09-10 17:04:02.349 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-27d30d50-83a3-4a2f-8ff9-e2b2a2e9190a - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:02.349 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-27d30d50-83a3-4a2f-8ff9-e2b2a2e9190a - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:02.391 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-27d30d50-83a3-4a2f-8ff9-e2b2a2e9190a - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:02.423 4076 3051093171896 MainThread INFO nova.compute.manager [req-27d30d50-83a3-4a2f-8ff9-e2b2a2e9190a - - - - -] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:02.502 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:04:02.763 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:02.831 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:04:02.969 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Updated VIF entry in instance network info cache for port bd224892-0bf2-4c64-a8c7-c69883004896. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:04:02.969 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Updating instance_info_cache with network_info: [{"id": "bd224892-0bf2-4c64-a8c7-c69883004896", "address": "fa:16:3e:85:b4:22", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbd224892-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:03.014 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-21240192-d5dc-4195-94d8-368b48995709 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-45310242-58a0-47b9-9655-a45945084513" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:03.014 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:03.028 4076 3051113163240 MainThread DEBUG nova.virt.hardware [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:04:03.028 4076 3051113163240 MainThread INFO nova.compute.claims [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Claim successful on node n-h1-808474-1 2021-09-10 17:04:03.034 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '15DCC8BE-F98A-41E9-A5E8-78F72E81CA18', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15290), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:03.143 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:03.143 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:03.165 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:03.165 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:03.165 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:03.165 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:04:03.180 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '15DCC8BE-F98A-41E9-A5E8-78F72E81CA18', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113859), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:03.180 4076 3051093170312 MainThread DEBUG os_win._utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9299890995025635 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:03.180 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-32d495c3-903d-4d66-935b-9e4af62bcbac - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:03.180 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-32d495c3-903d-4d66-935b-9e4af62bcbac - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:03.227 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-32d495c3-903d-4d66-935b-9e4af62bcbac - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:03.254 4076 3051055217128 MainThread INFO nova.compute.manager [req-32d495c3-903d-4d66-935b-9e4af62bcbac - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:03.375 4076 3051113163240 MainThread DEBUG nova.compute.provider_tree [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:03.375 4076 3051113163240 MainThread DEBUG nova.scheduler.client.report [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:03.375 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:03.375 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:04:03.482 4076 3051113163240 MainThread DEBUG nova.compute.utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:04:03.482 4076 3051055217128 GreenThread-82 DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:04:03.482 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:04:03.505 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:04:03.665 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:04:03.687 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Spawning new instance 2021-09-10 17:04:03.692 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:03.692 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:03.692 4076 3051113163240 MainThread DEBUG nova.block_device [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:04:03.692 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:03.692 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:03.706 4076 3051113163240 MainThread DEBUG os_win.utils.pathutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Creating directory: C:\openstack\instances\instance-00000060 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:04:03.706 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000060\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:04:03.720 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Resizing VHD C:\openstack\instances\instance-00000060\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:04:03.752 4076 3051055217128 GreenThread-82 DEBUG nova.policy [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3cd10bcdfa0441069cf30193fce4ef96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a35f4d0e56d743a8a248b3c564c45f4e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:04:04.204 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '83E1015F-8542-466F-802D-AC54EADA43CA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15057), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:04.273 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:04:04.273 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:04.273 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:04.273 4076 3051093170664 MainThread DEBUG os_win.utils.compute.vmutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-0000005c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:04:04.296 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-0000005c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:04:04.346 4076 3051093171896 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:04.346 4076 3051093171896 MainThread INFO nova.compute.manager [-] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] VM Stopped (Lifecycle Event) 2021-09-10 17:04:04.346 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:04.346 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:04.346 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '310082E2-2A9C-44DF-A184-4DFF59A1C405', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15183), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:04.346 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005d\' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)instance-0000005d5BF520B6-807A-47C2-ADE0-45E9466834EF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '83E1015F-8542-466F-802D-AC54EADA43CA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111314), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\nNot enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)\n\n'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF)", 'ErrorSummaryDescription': "'instance-0000005d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:04.346 4076 3051093170312 MainThread ERROR nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Failed to change vm state of instance-0000005d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. 2021-09-10 17:04:04.346 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Got request to destroy instance 2021-09-10 17:04:04.367 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-e3433eed-85fa-45ad-a025-8dd519355036 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:04.367 4076 3051093171896 MainThread DEBUG nova.virt.hyperv.vmops [req-e3433eed-85fa-45ad-a025-8dd519355036 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:04.417 4076 3051093171896 MainThread DEBUG nova.compute.manager [req-e3433eed-85fa-45ad-a025-8dd519355036 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:04.438 4076 3051093171896 MainThread INFO nova.compute.manager [req-e3433eed-85fa-45ad-a025-8dd519355036 - - - - -] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:04.454 4076 3051093170664 MainThread DEBUG os_win.utils.jobutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': '310082E2-2A9C-44DF-A184-4DFF59A1C405', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29123), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:04:04.454 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:04.454 4076 3051093170664 MainThread DEBUG os_win.utils.pathutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-0000005c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:04:04.476 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:04.476 4076 3051093170664 MainThread ERROR nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Traceback (most recent call last): 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(*args, **kwargs) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] admin_password, network_info, block_device_info) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(self, *args, **kwds) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] should_plug_vifs=False) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'req_state': req_state}) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] time.sleep(sleep_time) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return f(*args, **kwargs) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return self._wait_for_job(job_path) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] error_desc=err_desc) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During handling of the above exception, another exception occurred: 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Traceback (most recent call last): 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] yield resources 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] accel_info=accel_info) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(*args, **kwargs) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] admin_password, network_info, block_device_info) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(self, *args, **kwds) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] should_plug_vifs=False) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'req_state': req_state}) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] time.sleep(sleep_time) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return f(*args, **kwargs) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return self._wait_for_job(job_path) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] error_desc=err_desc) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.476 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.476 4076 3051093170664 MainThread INFO nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Terminating instance 2021-09-10 17:04:04.476 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:04:04.476 4076 3051093170664 MainThread INFO nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Got request to destroy instance 2021-09-10 17:04:04.496 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:04:04.496 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:04.496 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-0000005c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:04.496 4076 3051093170664 MainThread INFO nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:04:04.505 4076 3051093170664 MainThread DEBUG nova.compute.claims [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:04:04.505 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:04.703 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.hostops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:04.705 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.hostops [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:04.929 4076 3051093170664 MainThread DEBUG nova.compute.provider_tree [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG nova.scheduler.client.report [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:04.954 4076 3051093170664 MainThread ERROR nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Traceback (most recent call last): 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(*args, **kwargs) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] admin_password, network_info, block_device_info) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(self, *args, **kwds) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] should_plug_vifs=False) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'req_state': req_state}) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] time.sleep(sleep_time) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return f(*args, **kwargs) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return self._wait_for_job(job_path) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] error_desc=err_desc) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] During handling of the above exception, another exception occurred: 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Traceback (most recent call last): 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] accel_info=accel_info) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(*args, **kwargs) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] admin_password, network_info, block_device_info) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return function(self, *args, **kwds) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] should_plug_vifs=False) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'req_state': req_state}) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] time.sleep(sleep_time) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self.force_reraise() 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] raise self.value 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return f(*args, **kwargs) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] return self._wait_for_job(job_path) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] error_desc=err_desc) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. 2021-09-10 17:04:04.954 4076 3051093170664 MainThread TRACE nova.compute.manager [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG nova.compute.utils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Build of instance 442531a0-4fb7-4282-a436-20dfe4a05c72 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000005c' failed to start. Not enough memory in the system to start the virtual machine instance-0000005c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005c' failed to start. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Not enough memory in the system to start the virtual machine instance-0000005c with ram size 1024 megabytes. (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) 'instance-0000005c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A7F73905-E9A8-4792-A855-0C27BBC5BE02) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:04:04.954 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:04:04.970 4076 3051093170664 MainThread DEBUG nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:04:04.970 4076 3051093170664 MainThread DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:04:05.610 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Successfully created port: 56f4ea4e-a882-4a23-aece-82896714cfca _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:04:06.141 4076 3051093170664 MainThread DEBUG nova.network.neutron [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:06.193 4076 3051093170664 MainThread INFO nova.compute.manager [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 442531a0-4fb7-4282-a436-20dfe4a05c72] Took 1.22 seconds to deallocate network for instance. 2021-09-10 17:04:06.224 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:04:06.224 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:06.224 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:06.250 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Successfully changed vm state of instance-0000005d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:04:06.250 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Successfully changed state of VM instance-0000005d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:04:06.305 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Waiting for WMI job: {'InstanceID': '0D5AFDA5-B48E-4E9B-8177-7EEBE34115E5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20149), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:06.406 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] WMI job succeeded: {'InstanceID': '0D5AFDA5-B48E-4E9B-8177-7EEBE34115E5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28462), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:04:06.406 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:06.406 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Removing directory: C:\openstack\instances\instance-0000005d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:04:06.406 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:06.406 4076 3051093170312 MainThread ERROR nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Traceback (most recent call last): 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(*args, **kwargs) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] admin_password, network_info, block_device_info) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(self, *args, **kwds) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] should_plug_vifs=False) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'req_state': req_state}) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] time.sleep(sleep_time) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return f(*args, **kwargs) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return self._wait_for_job(job_path) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] error_desc=err_desc) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During handling of the above exception, another exception occurred: 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Traceback (most recent call last): 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] yield resources 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] accel_info=accel_info) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(*args, **kwargs) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] admin_password, network_info, block_device_info) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(self, *args, **kwds) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] should_plug_vifs=False) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'req_state': req_state}) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] time.sleep(sleep_time) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return f(*args, **kwargs) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return self._wait_for_job(job_path) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] error_desc=err_desc) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:06.406 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:06.406 4076 3051093170312 MainThread INFO nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Terminating instance 2021-09-10 17:04:06.406 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:04:06.406 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Got request to destroy instance 2021-09-10 17:04:06.439 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:04:06.439 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:06.439 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "instance-0000005d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:06.439 4076 3051093170312 MainThread INFO nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:04:06.439 4076 3051093170312 MainThread DEBUG nova.compute.claims [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:04:06.454 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:06.488 4076 3051093170664 MainThread INFO nova.scheduler.client.report [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance 442531a0-4fb7-4282-a436-20dfe4a05c72 2021-09-10 17:04:06.645 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [req-7e1dec41-105b-47f2-8e36-5624ef0b408e 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "442531a0-4fb7-4282-a436-20dfe4a05c72" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:06.787 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:06.805 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:07.023 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:07.044 4076 3051093170312 MainThread ERROR nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Traceback (most recent call last): 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(*args, **kwargs) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] admin_password, network_info, block_device_info) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(self, *args, **kwds) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] should_plug_vifs=False) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'req_state': req_state}) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] time.sleep(sleep_time) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return f(*args, **kwargs) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return self._wait_for_job(job_path) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] error_desc=err_desc) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] During handling of the above exception, another exception occurred: 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Traceback (most recent call last): 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] accel_info=accel_info) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(*args, **kwargs) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] admin_password, network_info, block_device_info) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return function(self, *args, **kwds) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] should_plug_vifs=False) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'req_state': req_state}) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] time.sleep(sleep_time) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self.force_reraise() 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] raise self.value 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return f(*args, **kwargs) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] return self._wait_for_job(job_path) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] error_desc=err_desc) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. 2021-09-10 17:04:07.044 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.compute.utils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Build of instance d856fdf2-ccbb-4aa6-8dbf-c62e4989911d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000005d' failed to start. Not enough memory in the system to start the virtual machine instance-0000005d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005d' failed to start. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Not enough memory in the system to start the virtual machine instance-0000005d with ram size 1024 megabytes. (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) 'instance-0000005d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5BF520B6-807A-47C2-ADE0-45E9466834EF) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:04:07.044 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:04:07.132 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-bfe6e759-174e-47d2-9674-6df76e9475bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Received event network-vif-plugged-bd224892-0bf2-4c64-a8c7-c69883004896 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:07.132 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-bfe6e759-174e-47d2-9674-6df76e9475bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "45310242-58a0-47b9-9655-a45945084513-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:07.132 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-bfe6e759-174e-47d2-9674-6df76e9475bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "45310242-58a0-47b9-9655-a45945084513-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:07.132 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-bfe6e759-174e-47d2-9674-6df76e9475bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Processing event network-vif-plugged-bd224892-0bf2-4c64-a8c7-c69883004896 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:04:07.187 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Using config drive for instance 2021-09-10 17:04:07.531 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Creating config drive at C:\openstack\instances\instance-0000005f\configdrive.iso 2021-09-10 17:04:07.554 4076 3051093170136 MainThread DEBUG oslo_concurrency.processutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps960o062 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:04:07.863 4076 3051093170136 MainThread DEBUG oslo_concurrency.processutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000005f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmps960o062" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:04:08.188 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:08.250 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-0000005f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:04:08.279 4076 3051093170312 MainThread INFO nova.compute.manager [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] [instance: d856fdf2-ccbb-4aa6-8dbf-c62e4989911d] Took 1.23 seconds to deallocate network for instance. 2021-09-10 17:04:08.298 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:04:08.333 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '3067D215-A32B-4C86-BB36-1F15B28D4EB4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14817), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:08.470 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:08.470 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:08.470 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:08.470 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:08.470 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3067D215-A32B-4C86-BB36-1F15B28D4EB4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113648), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:08.470 4076 3051093170136 MainThread DEBUG os_win._utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17238187789916992 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:08.504 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-de9e73ea-fdac-426b-9978-090ea2832265 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:08.504 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-de9e73ea-fdac-426b-9978-090ea2832265 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:08.551 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-de9e73ea-fdac-426b-9978-090ea2832265 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:08.578 4076 3051113164648 MainThread INFO nova.compute.manager [req-de9e73ea-fdac-426b-9978-090ea2832265 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:08.594 4076 3051093170312 MainThread INFO nova.scheduler.client.report [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Deleted allocations for instance d856fdf2-ccbb-4aa6-8dbf-c62e4989911d 2021-09-10 17:04:08.648 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-45b6e5f0-e23e-430d-8e4b-db2931e37309 206940e6bc1a478bb7f92f7778bec0e2 5283391577e14a77bbd03e5dcf9f9e5c - default default] Lock "d856fdf2-ccbb-4aa6-8dbf-c62e4989911d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.110s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:09.062 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Successfully updated port: 56f4ea4e-a882-4a23-aece-82896714cfca _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:04:09.086 4076 3051055217128 GreenThread-82 DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Acquired lock "refresh_cache-266ed819-0d7a-461a-91d1-188f0daec6fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:09.094 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:04:09.235 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Received event network-changed-56f4ea4e-a882-4a23-aece-82896714cfca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:09.235 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Refreshing instance network info cache due to event network-changed-56f4ea4e-a882-4a23-aece-82896714cfca. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:04:09.344 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:04:09.500 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '70DAEB93-8E21-4AFE-885C-23FCA73F05E7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15090), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:09.630 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:09.630 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:09.630 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:09.630 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:09.644 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '70DAEB93-8E21-4AFE-885C-23FCA73F05E7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109360), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:09.644 4076 3051093170136 MainThread DEBUG os_win._utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3457813262939453 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:09.664 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-e57fabef-75bf-4047-a42f-46bf7293ccf2 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:09.664 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-e57fabef-75bf-4047-a42f-46bf7293ccf2 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:09.703 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-e57fabef-75bf-4047-a42f-46bf7293ccf2 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:09.735 4076 3051093168552 MainThread INFO nova.compute.manager [req-e57fabef-75bf-4047-a42f-46bf7293ccf2 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:10.255 4076 3051055217128 GreenThread-82 DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Updating instance_info_cache with network_info: [{"id": "56f4ea4e-a882-4a23-aece-82896714cfca", "address": "fa:16:3e:10:da:04", "network": {"id": "5aedda23-2066-4b60-8615-f7279a31bfb7", "bridge": null, "label": "tempest-ServerTagsTestJSON-1743250206-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a35f4d0e56d743a8a248b3c564c45f4e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap56f4ea4e-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:10.275 4076 3051055217128 GreenThread-82 DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Releasing lock "refresh_cache-266ed819-0d7a-461a-91d1-188f0daec6fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:10.275 4076 3051055217128 GreenThread-82 DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Instance network_info: |[{"id": "56f4ea4e-a882-4a23-aece-82896714cfca", "address": "fa:16:3e:10:da:04", "network": {"id": "5aedda23-2066-4b60-8615-f7279a31bfb7", "bridge": null, "label": "tempest-ServerTagsTestJSON-1743250206-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a35f4d0e56d743a8a248b3c564c45f4e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap56f4ea4e-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:04:10.275 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Preparing to wait for external event network-vif-plugged-56f4ea4e-a882-4a23-aece-82896714cfca prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:04:10.275 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:10.275 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:10.275 4076 3051113163240 MainThread DEBUG os_win.utils.compute.vmutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Creating VM instance-00000060 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:04:10.275 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-266ed819-0d7a-461a-91d1-188f0daec6fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:10.275 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Refreshing network info cache for port 56f4ea4e-a882-4a23-aece-82896714cfca _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:04:10.674 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '1D5A5A1A-CE94-43FD-AA55-13A5AF60E094', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15071), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:10.804 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:10.804 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:10.815 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:10.815 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:10.815 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1D5A5A1A-CE94-43FD-AA55-13A5AF60E094', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110738), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:10.815 4076 3051093170136 MainThread DEBUG os_win._utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.517357587814331 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:10.831 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-be60b16e-83ba-4350-94e8-8f47e0a22a22 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:10.831 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-be60b16e-83ba-4350-94e8-8f47e0a22a22 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:10.876 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-be60b16e-83ba-4350-94e8-8f47e0a22a22 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:10.909 4076 3051093168552 MainThread INFO nova.compute.manager [req-be60b16e-83ba-4350-94e8-8f47e0a22a22 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:11.188 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:04:11.849 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '1D69ECDE-A005-4468-8DA2-F1309FE7785F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16922), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:11.977 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:11.977 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:11.977 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:11.977 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:11.977 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1D69ECDE-A005-4468-8DA2-F1309FE7785F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109903), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:11.977 4076 3051093170136 MainThread DEBUG os_win._utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6791775226593018 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:12.000 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-ea003858-45a1-4d90-aa1d-8eec55b00b05 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:12.000 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-ea003858-45a1-4d90-aa1d-8eec55b00b05 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:12.045 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-ea003858-45a1-4d90-aa1d-8eec55b00b05 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:12.062 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Updated VIF entry in instance network info cache for port 56f4ea4e-a882-4a23-aece-82896714cfca. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:04:12.062 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Updating instance_info_cache with network_info: [{"id": "56f4ea4e-a882-4a23-aece-82896714cfca", "address": "fa:16:3e:10:da:04", "network": {"id": "5aedda23-2066-4b60-8615-f7279a31bfb7", "bridge": null, "label": "tempest-ServerTagsTestJSON-1743250206-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "a35f4d0e56d743a8a248b3c564c45f4e", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap56f4ea4e-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:12.078 4076 3051093170312 MainThread INFO nova.compute.manager [req-ea003858-45a1-4d90-aa1d-8eec55b00b05 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:12.078 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-02f91d6c-70fd-45ea-82f6-2466bf990e78 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-266ed819-0d7a-461a-91d1-188f0daec6fa" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:13.015 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '5C8AC85F-E5D2-44EC-97DE-3718719197E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14811), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:13.145 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:13.145 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:13.145 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:13.145 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:13.145 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5C8AC85F-E5D2-44EC-97DE-3718719197E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111199), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:13.145 4076 3051093170136 MainThread DEBUG os_win._utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.847449779510498 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:13.172 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-c866bb64-1981-4898-8e63-4d07295149fb - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:13.172 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-c866bb64-1981-4898-8e63-4d07295149fb - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:13.218 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-c866bb64-1981-4898-8e63-4d07295149fb - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:13.250 4076 3051055217128 MainThread INFO nova.compute.manager [req-c866bb64-1981-4898-8e63-4d07295149fb - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:14.173 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '67F0C2E5-AAFA-4170-93E1-A2E6D2CCB39F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14670), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:14.304 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:14.304 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:14.304 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:14.317 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 45310242-58a0-47b9-9655-a45945084513] VM Stopped (Lifecycle Event) 2021-09-10 17:04:14.331 4076 3051093170136 MainThread ERROR os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000005f\' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000005f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)instance-0000005fF9D96D43-4337-40D4-A4B6-7AA208F2F9CE%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '67F0C2E5-AAFA-4170-93E1-A2E6D2CCB39F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109388), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\nNot enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)\n\n'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE)", 'ErrorSummaryDescription': "'instance-0000005f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000005f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:14.331 4076 3051093170136 MainThread ERROR nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-0000005f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. 2021-09-10 17:04:14.331 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Got request to destroy instance 2021-09-10 17:04:14.331 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-52b56f30-6f13-4643-85bf-38ace8cfb715 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:14.331 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-52b56f30-6f13-4643-85bf-38ace8cfb715 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:14.390 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-52b56f30-6f13-4643-85bf-38ace8cfb715 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:14.426 4076 3051055217128 MainThread INFO nova.compute.manager [req-52b56f30-6f13-4643-85bf-38ace8cfb715 - - - - -] [instance: 45310242-58a0-47b9-9655-a45945084513] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:15.711 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:15.735 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:04:15.895 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:15.895 4076 3051093170312 MainThread DEBUG nova.virt.hardware [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:04:15.908 4076 3051093170312 MainThread INFO nova.compute.claims [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Claim successful on node n-h1-808474-1 2021-09-10 17:04:16.054 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:16.065 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:16.208 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:04:16.208 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.208 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.226 4076 3051093170136 MainThread DEBUG os_win.utils.compute.vmutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-0000005f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:04:16.238 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-0000005f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:04:16.274 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:16.288 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'EBD483AB-CC91-49B0-B58F-2DFC962EC94A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14939), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:16.297 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:16.297 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.297 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:04:16.396 4076 3051093170312 MainThread DEBUG nova.compute.utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:04:16.396 4076 3051055217128 GreenThread-83 DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:04:16.396 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:04:16.396 4076 3051093170136 MainThread DEBUG os_win.utils.jobutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': 'EBD483AB-CC91-49B0-B58F-2DFC962EC94A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30699), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:04:16.396 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.396 4076 3051093170136 MainThread DEBUG os_win.utils.pathutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-0000005f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:04:16.396 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.396 4076 3051093170136 MainThread ERROR nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Traceback (most recent call last): 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(*args, **kwargs) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] admin_password, network_info, block_device_info) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(self, *args, **kwds) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] should_plug_vifs=False) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'req_state': req_state}) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] time.sleep(sleep_time) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return f(*args, **kwargs) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return self._wait_for_job(job_path) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] error_desc=err_desc) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] During handling of the above exception, another exception occurred: 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Traceback (most recent call last): 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] yield resources 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] accel_info=accel_info) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(*args, **kwargs) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] admin_password, network_info, block_device_info) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(self, *args, **kwds) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] should_plug_vifs=False) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'req_state': req_state}) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] time.sleep(sleep_time) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return f(*args, **kwargs) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return self._wait_for_job(job_path) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] error_desc=err_desc) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.396 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.423 4076 3051093170136 MainThread INFO nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Terminating instance 2021-09-10 17:04:16.423 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:04:16.423 4076 3051093170136 MainThread INFO nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Got request to destroy instance 2021-09-10 17:04:16.423 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:04:16.438 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:04:16.438 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.438 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000005f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.438 4076 3051093170136 MainThread INFO nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:04:16.438 4076 3051093170136 MainThread DEBUG nova.compute.claims [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:04:16.438 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.626 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:04:16.626 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Spawning new instance 2021-09-10 17:04:16.647 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:16.647 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.647 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.647 4076 3051093170312 MainThread DEBUG nova.block_device [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:04:16.647 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:16.647 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.656 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Creating directory: C:\openstack\instances\instance-00000061 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:04:16.656 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000061\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:04:16.656 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.hostops [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:16.672 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Resizing VHD C:\openstack\instances\instance-00000061\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:04:16.783 4076 3051055217128 GreenThread-83 DEBUG nova.policy [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '30f08d002bf64c7db4d91135559d5b8d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e40c2b3a02a2453b8747f81262f1ddbd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:04:16.866 4076 3051093170136 MainThread DEBUG nova.compute.provider_tree [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:16.882 4076 3051093170136 MainThread DEBUG nova.scheduler.client.report [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:16.882 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:16.898 4076 3051093170136 MainThread ERROR nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Traceback (most recent call last): 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(*args, **kwargs) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] admin_password, network_info, block_device_info) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(self, *args, **kwds) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] should_plug_vifs=False) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'req_state': req_state}) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] time.sleep(sleep_time) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return f(*args, **kwargs) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return self._wait_for_job(job_path) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] error_desc=err_desc) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] During handling of the above exception, another exception occurred: 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Traceback (most recent call last): 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] accel_info=accel_info) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(*args, **kwargs) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] admin_password, network_info, block_device_info) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return function(self, *args, **kwds) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] should_plug_vifs=False) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'req_state': req_state}) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] time.sleep(sleep_time) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self.force_reraise() 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] raise self.value 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return f(*args, **kwargs) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] return self._wait_for_job(job_path) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] error_desc=err_desc) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. 2021-09-10 17:04:16.898 4076 3051093170136 MainThread TRACE nova.compute.manager [instance: 45310242-58a0-47b9-9655-a45945084513] 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.compute.utils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Build of instance 45310242-58a0-47b9-9655-a45945084513 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000005f' failed to start. Not enough memory in the system to start the virtual machine instance-0000005f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000005f' failed to start. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Not enough memory in the system to start the virtual machine instance-0000005f with ram size 1024 megabytes. (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) 'instance-0000005f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F9D96D43-4337-40D4-A4B6-7AA208F2F9CE) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:04:16.898 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:04:17.619 4076 3051093170136 MainThread DEBUG nova.network.neutron [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:17.656 4076 3051093170136 MainThread INFO nova.compute.manager [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 45310242-58a0-47b9-9655-a45945084513] Took 0.73 seconds to deallocate network for instance. 2021-09-10 17:04:17.913 4076 3051093170136 MainThread INFO nova.scheduler.client.report [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance 45310242-58a0-47b9-9655-a45945084513 2021-09-10 17:04:17.953 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-851518f1-29ec-48a0-881c-3c234ff9ede6 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "45310242-58a0-47b9-9655-a45945084513" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 22.922s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:18.845 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Successfully created port: 0a9574bb-cc00-4128-9ce4-05984253986a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:04:18.954 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-ca80a570-ddae-4357-9274-8eb285c3ccbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Received event network-vif-plugged-56f4ea4e-a882-4a23-aece-82896714cfca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:18.984 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-ca80a570-ddae-4357-9274-8eb285c3ccbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:18.984 4076 3051093170136 MainThread DEBUG oslo_concurrency.lockutils [req-ca80a570-ddae-4357-9274-8eb285c3ccbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:18.984 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-ca80a570-ddae-4357-9274-8eb285c3ccbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Processing event network-vif-plugged-56f4ea4e-a882-4a23-aece-82896714cfca _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:04:19.054 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Using config drive for instance 2021-09-10 17:04:19.288 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Creating config drive at C:\openstack\instances\instance-00000060\configdrive.iso 2021-09-10 17:04:19.317 4076 3051113163240 MainThread DEBUG oslo_concurrency.processutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000060\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8htr7wtq execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:04:19.625 4076 3051113163240 MainThread DEBUG oslo_concurrency.processutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000060\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8htr7wtq" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:04:20.001 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Setting boot order for instance: instance-00000060: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:04:20.056 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:04:20.062 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': '861F0871-527F-4738-8F32-5E4CB41A013B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14226), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:20.210 4076 3051093170136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:20.210 4076 3051093170136 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:20.210 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:20.229 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:20.333 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '861F0871-527F-4738-8F32-5E4CB41A013B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112956), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:20.333 4076 3051113163240 MainThread DEBUG os_win._utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.2779068946838379 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:20.352 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-aef12250-55f9-4931-9db3-f40c58059e1f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:20.352 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-aef12250-55f9-4931-9db3-f40c58059e1f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:20.396 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-aef12250-55f9-4931-9db3-f40c58059e1f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:20.428 4076 3051093170136 MainThread INFO nova.compute.manager [req-aef12250-55f9-4931-9db3-f40c58059e1f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:21.348 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': '83D62A31-E4AF-403A-8103-64F2EA5F7CDA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15118), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:21.469 4076 3051093170136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:21.469 4076 3051093170136 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:21.469 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:21.493 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:21.503 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '83D62A31-E4AF-403A-8103-64F2EA5F7CDA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108548), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:21.503 4076 3051113163240 MainThread DEBUG os_win._utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4469537734985352 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:21.503 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-4c9eaa43-5395-4b07-a7c5-b1bc97d20dcf - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:21.503 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-4c9eaa43-5395-4b07-a7c5-b1bc97d20dcf - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:21.555 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-4c9eaa43-5395-4b07-a7c5-b1bc97d20dcf - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:21.564 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Successfully updated port: 0a9574bb-cc00-4128-9ce4-05984253986a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:04:21.594 4076 3051055217128 GreenThread-83 DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Acquired lock "refresh_cache-7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:21.594 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:04:21.594 4076 3051093170136 MainThread INFO nova.compute.manager [req-4c9eaa43-5395-4b07-a7c5-b1bc97d20dcf - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:21.756 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:04:21.813 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Received event network-changed-0a9574bb-cc00-4128-9ce4-05984253986a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:21.813 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Refreshing instance network info cache due to event network-changed-0a9574bb-cc00-4128-9ce4-05984253986a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:04:22.495 4076 3051055217128 GreenThread-83 DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Updating instance_info_cache with network_info: [{"id": "0a9574bb-cc00-4128-9ce4-05984253986a", "address": "fa:16:3e:9c:e2:36", "network": {"id": "94fe800d-2f58-4038-bad7-4a550b92b7da", "bridge": null, "label": "tempest-ServerAddressesNegativeTestJSON-129404465-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e40c2b3a02a2453b8747f81262f1ddbd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0a9574bb-cc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:22.514 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': 'EA910060-63A6-47D1-9E2A-242C31E76440', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17434), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:22.531 4076 3051055217128 GreenThread-83 DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Releasing lock "refresh_cache-7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:22.531 4076 3051055217128 GreenThread-83 DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Instance network_info: |[{"id": "0a9574bb-cc00-4128-9ce4-05984253986a", "address": "fa:16:3e:9c:e2:36", "network": {"id": "94fe800d-2f58-4038-bad7-4a550b92b7da", "bridge": null, "label": "tempest-ServerAddressesNegativeTestJSON-129404465-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e40c2b3a02a2453b8747f81262f1ddbd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0a9574bb-cc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:04:22.537 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Preparing to wait for external event network-vif-plugged-0a9574bb-cc00-4128-9ce4-05984253986a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:04:22.537 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:22.537 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:22.537 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Creating VM instance-00000061 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:04:22.537 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:22.537 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Refreshing network info cache for port 0a9574bb-cc00-4128-9ce4-05984253986a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:04:22.653 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:22.653 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:22.653 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:22.653 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:22.663 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EA910060-63A6-47D1-9E2A-242C31E76440', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111574), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:22.663 4076 3051113163240 MainThread DEBUG os_win._utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.607900381088257 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:22.679 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-d9554cc6-c285-41da-a668-4100fbb3c9a2 - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:22.679 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-d9554cc6-c285-41da-a668-4100fbb3c9a2 - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:22.735 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-d9554cc6-c285-41da-a668-4100fbb3c9a2 - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:22.781 4076 3051113164648 MainThread INFO nova.compute.manager [req-d9554cc6-c285-41da-a668-4100fbb3c9a2 - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:23.428 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:04:23.696 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': '9C6F3D5C-9759-4326-A9A4-171E737D8972', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15268), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:23.812 4076 3051093170136 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:23.834 4076 3051093170136 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:23.834 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:23.834 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:23.834 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9C6F3D5C-9759-4326-A9A4-171E737D8972', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111433), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:23.834 4076 3051113163240 MainThread DEBUG os_win._utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.778110980987549 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:23.834 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fc8e6d68-d22b-452c-ba0a-66e4d0cc769e - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:23.834 4076 3051093170136 MainThread DEBUG nova.virt.hyperv.vmops [req-fc8e6d68-d22b-452c-ba0a-66e4d0cc769e - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:23.906 4076 3051093170136 MainThread DEBUG nova.compute.manager [req-fc8e6d68-d22b-452c-ba0a-66e4d0cc769e - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:23.934 4076 3051093170136 MainThread INFO nova.compute.manager [req-fc8e6d68-d22b-452c-ba0a-66e4d0cc769e - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:24.203 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Updated VIF entry in instance network info cache for port 0a9574bb-cc00-4128-9ce4-05984253986a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:04:24.203 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Updating instance_info_cache with network_info: [{"id": "0a9574bb-cc00-4128-9ce4-05984253986a", "address": "fa:16:3e:9c:e2:36", "network": {"id": "94fe800d-2f58-4038-bad7-4a550b92b7da", "bridge": null, "label": "tempest-ServerAddressesNegativeTestJSON-129404465-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "e40c2b3a02a2453b8747f81262f1ddbd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0a9574bb-cc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:24.218 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-c8db219a-9509-4a2b-bde2-9f2941f5ab35 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:24.844 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': 'D5E54C03-FBD3-413D-A8B0-BD19CEC8D657', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17506), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:24.989 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:24.989 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:24.989 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:24.989 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:24.989 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D5E54C03-FBD3-413D-A8B0-BD19CEC8D657', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110703), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:24.989 4076 3051113163240 MainThread DEBUG os_win._utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.933297157287598 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:25.015 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-60b79f9b-4abe-4cc7-a532-f98fd7871d4b - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:25.016 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-60b79f9b-4abe-4cc7-a532-f98fd7871d4b - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:25.034 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-60b79f9b-4abe-4cc7-a532-f98fd7871d4b - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:25.080 4076 3051055217128 MainThread INFO nova.compute.manager [req-60b79f9b-4abe-4cc7-a532-f98fd7871d4b - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:26.001 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': 'EFC6773F-E840-4511-84A5-845245F4DD0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15417), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:26.148 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:26.148 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:26.148 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:26.148 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] VM Stopped (Lifecycle Event) 2021-09-10 17:04:26.174 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job failed: {'RawErrors': (('124\'instance-00000060\' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000060\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)instance-00000060F5795397-FF5D-4455-B4BB-7074D155886B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EFC6773F-E840-4511-84A5-845245F4DD0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111882), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\nNot enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)\n\n'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B)", 'ErrorSummaryDescription': "'instance-00000060' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000060.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:26.174 4076 3051113163240 MainThread ERROR nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Failed to change vm state of instance-00000060 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. 2021-09-10 17:04:26.174 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Got request to destroy instance 2021-09-10 17:04:26.174 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-f17865c9-d748-4f5a-b4e7-f876fdb7cd5f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:26.174 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-f17865c9-d748-4f5a-b4e7-f876fdb7cd5f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:26.224 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-f17865c9-d748-4f5a-b4e7-f876fdb7cd5f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:26.257 4076 3051055217128 MainThread INFO nova.compute.manager [req-f17865c9-d748-4f5a-b4e7-f876fdb7cd5f - - - - -] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:28.066 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:04:28.066 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:28.066 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:28.080 4076 3051113163240 MainThread DEBUG os_win.utils.compute.vmutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Successfully changed vm state of instance-00000060 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:04:28.080 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Successfully changed state of VM instance-00000060 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:04:28.132 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Waiting for WMI job: {'InstanceID': 'E204D541-F1E5-4337-B801-DC8ED32B8570', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15914), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:28.257 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] WMI job succeeded: {'InstanceID': 'E204D541-F1E5-4337-B801-DC8ED32B8570', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29524), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:04:28.257 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:28.257 4076 3051113163240 MainThread DEBUG os_win.utils.pathutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Removing directory: C:\openstack\instances\instance-00000060 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:04:28.257 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:28.257 4076 3051113163240 MainThread ERROR nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Traceback (most recent call last): 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(*args, **kwargs) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] admin_password, network_info, block_device_info) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(self, *args, **kwds) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] should_plug_vifs=False) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'req_state': req_state}) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] time.sleep(sleep_time) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return f(*args, **kwargs) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return self._wait_for_job(job_path) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] error_desc=err_desc) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During handling of the above exception, another exception occurred: 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Traceback (most recent call last): 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] yield resources 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] accel_info=accel_info) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(*args, **kwargs) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] admin_password, network_info, block_device_info) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(self, *args, **kwds) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] should_plug_vifs=False) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'req_state': req_state}) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] time.sleep(sleep_time) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return f(*args, **kwargs) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return self._wait_for_job(job_path) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] error_desc=err_desc) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.257 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.257 4076 3051113163240 MainThread INFO nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Terminating instance 2021-09-10 17:04:28.257 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:04:28.257 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Got request to destroy instance 2021-09-10 17:04:28.281 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:04:28.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:28.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "instance-00000060" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:28.295 4076 3051113163240 MainThread INFO nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:04:28.295 4076 3051113163240 MainThread DEBUG nova.compute.claims [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:04:28.295 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:28.431 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:28.453 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:28.640 4076 3051113163240 MainThread DEBUG nova.compute.provider_tree [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:28.671 4076 3051113163240 MainThread DEBUG nova.scheduler.client.report [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:28.671 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:28.671 4076 3051113163240 MainThread ERROR nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Traceback (most recent call last): 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(*args, **kwargs) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] admin_password, network_info, block_device_info) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(self, *args, **kwds) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] should_plug_vifs=False) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'req_state': req_state}) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] time.sleep(sleep_time) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return f(*args, **kwargs) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return self._wait_for_job(job_path) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] error_desc=err_desc) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] During handling of the above exception, another exception occurred: 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Traceback (most recent call last): 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] accel_info=accel_info) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(*args, **kwargs) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] admin_password, network_info, block_device_info) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return function(self, *args, **kwds) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] should_plug_vifs=False) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'req_state': req_state}) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] time.sleep(sleep_time) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self.force_reraise() 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] raise self.value 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return f(*args, **kwargs) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] return self._wait_for_job(job_path) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] error_desc=err_desc) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. 2021-09-10 17:04:28.671 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.compute.utils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Build of instance 266ed819-0d7a-461a-91d1-188f0daec6fa was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000060' failed to start. Not enough memory in the system to start the virtual machine instance-00000060. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000060' failed to start. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Not enough memory in the system to start the virtual machine instance-00000060 with ram size 1024 megabytes. (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) 'instance-00000060' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F5795397-FF5D-4455-B4BB-7074D155886B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:04:28.687 4076 3051113163240 MainThread DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:04:29.554 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-5a7f7563-9cea-408f-a863-016bd48e1bff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Received event network-vif-plugged-0a9574bb-cc00-4128-9ce4-05984253986a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:29.554 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-5a7f7563-9cea-408f-a863-016bd48e1bff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:29.554 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-5a7f7563-9cea-408f-a863-016bd48e1bff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:29.554 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-5a7f7563-9cea-408f-a863-016bd48e1bff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Processing event network-vif-plugged-0a9574bb-cc00-4128-9ce4-05984253986a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:04:29.597 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Using config drive for instance 2021-09-10 17:04:29.735 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Creating config drive at C:\openstack\instances\instance-00000061\configdrive.iso 2021-09-10 17:04:29.788 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000061\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp23py72i0 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:04:29.952 4076 3051113163240 MainThread DEBUG nova.network.neutron [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:29.956 4076 3051113163240 MainThread INFO nova.compute.manager [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] [instance: 266ed819-0d7a-461a-91d1-188f0daec6fa] Took 1.27 seconds to deallocate network for instance. 2021-09-10 17:04:30.096 4076 3051093170312 MainThread DEBUG oslo_concurrency.processutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000061\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp23py72i0" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:04:30.192 4076 3051113163240 MainThread INFO nova.scheduler.client.report [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Deleted allocations for instance 266ed819-0d7a-461a-91d1-188f0daec6fa 2021-09-10 17:04:30.267 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-0203215f-f540-4d7c-8ad9-e81cc8b5f1d2 3cd10bcdfa0441069cf30193fce4ef96 a35f4d0e56d743a8a248b3c564c45f4e - default default] Lock "266ed819-0d7a-461a-91d1-188f0daec6fa" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:30.471 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Setting boot order for instance: instance-00000061: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:04:30.520 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:04:30.556 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '725BD8D2-47E9-4B09-B6A7-95DC434DB22B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14798), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:30.703 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:30.704 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:30.704 4076 3051113163240 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:30.704 4076 3051113163240 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:30.704 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '725BD8D2-47E9-4B09-B6A7-95DC434DB22B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111659), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:30.704 4076 3051093170312 MainThread DEBUG os_win._utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1839909553527832 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:30.726 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-9ccbc77d-850f-4706-b623-d78a78e8a920 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:30.726 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-9ccbc77d-850f-4706-b623-d78a78e8a920 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:30.765 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-9ccbc77d-850f-4706-b623-d78a78e8a920 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:30.797 4076 3051113163240 MainThread INFO nova.compute.manager [req-9ccbc77d-850f-4706-b623-d78a78e8a920 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:31.744 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '4A400CA8-C5ED-4312-983E-2B74EC8EB9D4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15218), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:31.879 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:31.879 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:31.879 4076 3051113163240 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:31.879 4076 3051113163240 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:31.898 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-e7af5ac4-a2bc-4303-9a37-2b1f234eaafe - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:31.898 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-e7af5ac4-a2bc-4303-9a37-2b1f234eaafe - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:31.898 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4A400CA8-C5ED-4312-983E-2B74EC8EB9D4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111552), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:31.898 4076 3051093170312 MainThread DEBUG os_win._utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3777194023132324 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:31.941 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-e7af5ac4-a2bc-4303-9a37-2b1f234eaafe - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:31.984 4076 3051113163240 MainThread INFO nova.compute.manager [req-e7af5ac4-a2bc-4303-9a37-2b1f234eaafe - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:32.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:32.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:04:32.914 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '3964501D-95BC-41D7-B2C8-C7378DE7B0D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16614), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:33.050 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:33.050 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:33.050 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:33.050 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:33.050 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3964501D-95BC-41D7-B2C8-C7378DE7B0D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109465), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:33.050 4076 3051093170312 MainThread DEBUG os_win._utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.53025484085083 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:33.063 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:33.063 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:04:33.081 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-c7207c08-318f-4a22-ad58-42145980812d - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:33.081 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-c7207c08-318f-4a22-ad58-42145980812d - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:33.137 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-c7207c08-318f-4a22-ad58-42145980812d - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:33.173 4076 3051113160952 MainThread INFO nova.compute.manager [req-c7207c08-318f-4a22-ad58-42145980812d - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:34.082 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '797235AF-E4DE-4AC7-BD39-2BDA750ED2BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15673), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:34.228 4076 3051113164472 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:34.228 4076 3051113164472 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:34.228 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:34.228 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:34.245 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '797235AF-E4DE-4AC7-BD39-2BDA750ED2BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114699), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:34.245 4076 3051093170312 MainThread DEBUG os_win._utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.724801778793335 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:34.284 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-7c241956-f7cb-4810-8314-30e241633544 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:34.284 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [req-7c241956-f7cb-4810-8314-30e241633544 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:34.312 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-7c241956-f7cb-4810-8314-30e241633544 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:34.402 4076 3051113164472 MainThread INFO nova.compute.manager [req-7c241956-f7cb-4810-8314-30e241633544 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:35.188 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:35.218 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:35.218 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:04:35.218 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:35.273 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '31EBC515-237A-47AE-9AC2-47F255A1143C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14627), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:35.406 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:35.406 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:35.414 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:35.414 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:35.414 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '31EBC515-237A-47AE-9AC2-47F255A1143C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110943), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:35.414 4076 3051093170312 MainThread DEBUG os_win._utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.893730640411377 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:04:35.437 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-c169682c-be34-4319-8f1e-4936662498c0 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:35.437 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-c169682c-be34-4319-8f1e-4936662498c0 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:35.476 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-c169682c-be34-4319-8f1e-4936662498c0 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:35.499 4076 3051113163592 MainThread INFO nova.compute.manager [req-c169682c-be34-4319-8f1e-4936662498c0 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:36.437 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': '0ED87181-9C8D-4513-A02C-E4B1D2547765', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14385), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:36.569 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:36.569 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:36.569 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:04:36.569 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] VM Stopped (Lifecycle Event) 2021-09-10 17:04:36.569 4076 3051093170312 MainThread ERROR os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job failed: {'RawErrors': (('124\'instance-00000061\' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000061\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)instance-0000006171133D34-AEE9-4A5B-9349-7008189F9BA4%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0ED87181-9C8D-4513-A02C-E4B1D2547765', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113173), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 4, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 4, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\nNot enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)\n\n'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4)", 'ErrorSummaryDescription': "'instance-00000061' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000061.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:04:36.591 4076 3051093170312 MainThread ERROR nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Failed to change vm state of instance-00000061 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. 2021-09-10 17:04:36.591 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Got request to destroy instance 2021-09-10 17:04:36.596 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-e718e6a1-b791-40e5-9900-8575cbf04347 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:04:36.596 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-e718e6a1-b791-40e5-9900-8575cbf04347 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:04:36.632 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-e718e6a1-b791-40e5-9900-8575cbf04347 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:04:36.689 4076 3051113163592 MainThread INFO nova.compute.manager [req-e718e6a1-b791-40e5-9900-8575cbf04347 - - - - -] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:04:38.401 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:04:38.407 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:38.407 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:38.407 4076 3051093170312 MainThread DEBUG os_win.utils.compute.vmutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Successfully changed vm state of instance-00000061 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:04:38.407 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Successfully changed state of VM instance-00000061 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:04:38.455 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Waiting for WMI job: {'InstanceID': 'EBFFF15C-9D22-476D-8D59-E79114C05EE8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14175), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:04:38.579 4076 3051093170312 MainThread DEBUG os_win.utils.jobutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] WMI job succeeded: {'InstanceID': 'EBFFF15C-9D22-476D-8D59-E79114C05EE8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28590), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:04:38.579 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:38.579 4076 3051093170312 MainThread DEBUG os_win.utils.pathutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Removing directory: C:\openstack\instances\instance-00000061 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:04:38.579 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:38.579 4076 3051093170312 MainThread ERROR nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Traceback (most recent call last): 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(*args, **kwargs) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] admin_password, network_info, block_device_info) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(self, *args, **kwds) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] should_plug_vifs=False) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'req_state': req_state}) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] time.sleep(sleep_time) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return f(*args, **kwargs) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return self._wait_for_job(job_path) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] error_desc=err_desc) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During handling of the above exception, another exception occurred: 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Traceback (most recent call last): 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] yield resources 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] accel_info=accel_info) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(*args, **kwargs) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] admin_password, network_info, block_device_info) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(self, *args, **kwds) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] should_plug_vifs=False) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'req_state': req_state}) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] time.sleep(sleep_time) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return f(*args, **kwargs) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return self._wait_for_job(job_path) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] error_desc=err_desc) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.579 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.579 4076 3051093170312 MainThread INFO nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Terminating instance 2021-09-10 17:04:38.579 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:04:38.579 4076 3051093170312 MainThread INFO nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Got request to destroy instance 2021-09-10 17:04:38.609 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:04:38.609 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:38.609 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "instance-00000061" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:38.609 4076 3051093170312 MainThread INFO nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:04:38.609 4076 3051093170312 MainThread DEBUG nova.compute.claims [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:04:38.609 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:38.749 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:38.757 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.hostops [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:38.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:38.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:38.859 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:38.975 4076 3051093170312 MainThread DEBUG nova.compute.provider_tree [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:38.999 4076 3051093170312 MainThread DEBUG nova.scheduler.client.report [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:38.999 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:38.999 4076 3051093170312 MainThread ERROR nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Traceback (most recent call last): 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(*args, **kwargs) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] admin_password, network_info, block_device_info) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(self, *args, **kwds) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] should_plug_vifs=False) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'req_state': req_state}) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] time.sleep(sleep_time) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return f(*args, **kwargs) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return self._wait_for_job(job_path) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] error_desc=err_desc) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] During handling of the above exception, another exception occurred: 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Traceback (most recent call last): 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] accel_info=accel_info) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(*args, **kwargs) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] admin_password, network_info, block_device_info) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return function(self, *args, **kwds) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] should_plug_vifs=False) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'req_state': req_state}) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] time.sleep(sleep_time) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self.force_reraise() 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] raise self.value 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return f(*args, **kwargs) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] return self._wait_for_job(job_path) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] error_desc=err_desc) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. 2021-09-10 17:04:38.999 4076 3051093170312 MainThread TRACE nova.compute.manager [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] 2021-09-10 17:04:38.999 4076 3051093170312 MainThread DEBUG nova.compute.utils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:04:38.999 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Build of instance 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000061' failed to start. Not enough memory in the system to start the virtual machine instance-00000061. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000061' failed to start. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Not enough memory in the system to start the virtual machine instance-00000061 with ram size 1024 megabytes. (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) 'instance-00000061' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 71133D34-AEE9-4A5B-9349-7008189F9BA4) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:04:39.015 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:04:39.015 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:04:39.015 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:04:39.015 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:04:39.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:40.237 4076 3051093170312 MainThread DEBUG nova.network.neutron [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:40.280 4076 3051093170312 MainThread INFO nova.compute.manager [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] [instance: 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269] Took 1.27 seconds to deallocate network for instance. 2021-09-10 17:04:40.690 4076 3051093170312 MainThread INFO nova.scheduler.client.report [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Deleted allocations for instance 7bb24cc6-9fd0-42d3-8abb-a946f1ebc269 2021-09-10 17:04:40.763 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [req-a4793500-f986-4d94-9634-c0309828872c 30f08d002bf64c7db4d91135559d5b8d e40c2b3a02a2453b8747f81262f1ddbd - default default] Lock "7bb24cc6-9fd0-42d3-8abb-a946f1ebc269" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.063s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:41.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:41.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:41.813 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:41.813 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:41.813 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:04:41.813 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:41.831 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:42.048 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2186MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:04:42.048 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:42.273 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:42.304 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:04:42.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 87008c7c-0917-4c78-93e1-b929cb87a160 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 17:04:42.359 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:04:42.359 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:04:42.452 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:42.468 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:42.683 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:42.711 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:42.711 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:04:42.711 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.656s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:42.711 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.172s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:42.711 4076 3051113162712 MainThread DEBUG nova.virt.hardware [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:04:42.711 4076 3051113162712 MainThread INFO nova.compute.claims [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Claim successful on node n-h1-808474-1 2021-09-10 17:04:42.968 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.hostops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:42.984 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.hostops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:43.187 4076 3051113162712 MainThread DEBUG nova.compute.provider_tree [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:43.187 4076 3051113162712 MainThread DEBUG nova.scheduler.client.report [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:43.187 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:43.187 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:04:43.303 4076 3051113162712 MainThread DEBUG nova.compute.utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:04:43.303 4076 3051093170312 GreenThread-84 DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:04:43.303 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:04:43.341 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:04:43.489 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:04:43.489 4076 3051113162712 MainThread INFO nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Spawning new instance 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG nova.block_device [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG os_win.utils.pathutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-00000062 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:04:43.515 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000062\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:04:43.546 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Resizing VHD C:\openstack\instances\instance-00000062\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:04:43.695 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:43.695 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:04:43.742 4076 3051093170312 GreenThread-84 DEBUG nova.policy [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:04:44.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:04:46.660 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Successfully created port: 7c864e7f-dc26-4369-83b5-865dcfd95668 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:04:49.578 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Successfully updated port: 7c864e7f-dc26-4369-83b5-865dcfd95668 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:04:49.616 4076 3051093170312 GreenThread-84 DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-87008c7c-0917-4c78-93e1-b929cb87a160" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:49.616 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:04:49.783 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Received event network-changed-7c864e7f-dc26-4369-83b5-865dcfd95668 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:49.783 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Refreshing instance network info cache due to event network-changed-7c864e7f-dc26-4369-83b5-865dcfd95668. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:04:50.000 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:04:51.077 4076 3051093170312 GreenThread-84 DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Updating instance_info_cache with network_info: [{"id": "7c864e7f-dc26-4369-83b5-865dcfd95668", "address": "fa:16:3e:69:2d:14", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7c864e7f-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:51.106 4076 3051093170312 GreenThread-84 DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-87008c7c-0917-4c78-93e1-b929cb87a160" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:51.109 4076 3051093170312 GreenThread-84 DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Instance network_info: |[{"id": "7c864e7f-dc26-4369-83b5-865dcfd95668", "address": "fa:16:3e:69:2d:14", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7c864e7f-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:04:51.109 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Preparing to wait for external event network-vif-plugged-7c864e7f-dc26-4369-83b5-865dcfd95668 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:04:51.109 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:51.109 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:51.109 4076 3051113162712 MainThread DEBUG os_win.utils.compute.vmutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-00000062 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:04:51.109 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-87008c7c-0917-4c78-93e1-b929cb87a160" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:04:51.109 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Refreshing network info cache for port 7c864e7f-dc26-4369-83b5-865dcfd95668 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:04:51.972 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:04:52.529 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Updated VIF entry in instance network info cache for port 7c864e7f-dc26-4369-83b5-865dcfd95668. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:04:52.529 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Updating instance_info_cache with network_info: [{"id": "7c864e7f-dc26-4369-83b5-865dcfd95668", "address": "fa:16:3e:69:2d:14", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7c864e7f-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:04:52.557 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-d047c673-c50d-415d-bbaa-2264597c6f96 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-87008c7c-0917-4c78-93e1-b929cb87a160" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:04:57.867 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:57.889 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:04:58.022 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:58.022 4076 3051113161128 MainThread DEBUG nova.virt.hardware [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:04:58.022 4076 3051113161128 MainThread INFO nova.compute.claims [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Claim successful on node n-h1-808474-1 2021-09-10 17:04:58.179 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:04:58.204 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:04:58.399 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:04:58.425 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:04:58.425 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:58.425 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:04:58.561 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:04:58.561 4076 3051093170312 GreenThread-85 DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:04:58.561 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:04:58.597 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:04:58.806 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:04:58.827 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Spawning new instance 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG nova.block_device [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating directory: C:\openstack\instances\instance-00000064 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:04:58.836 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000064\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:04:58.874 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Resizing VHD C:\openstack\instances\instance-00000064\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:04:59.080 4076 3051093170312 GreenThread-85 DEBUG nova.policy [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6da982fe4daa44b998cd4ae6900212fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0928f57ccc9245e28c79dd9cce7e395b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:04:59.796 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-4d78554c-f892-4f0c-9e76-adfe386ce25c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Received event network-vif-plugged-7c864e7f-dc26-4369-83b5-865dcfd95668 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:04:59.796 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-4d78554c-f892-4f0c-9e76-adfe386ce25c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:04:59.796 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [req-4d78554c-f892-4f0c-9e76-adfe386ce25c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:04:59.796 4076 3051114314392 MainThread DEBUG nova.compute.manager [req-4d78554c-f892-4f0c-9e76-adfe386ce25c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Processing event network-vif-plugged-7c864e7f-dc26-4369-83b5-865dcfd95668 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:04:59.842 4076 3051113162712 MainThread INFO nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Using config drive for instance 2021-09-10 17:04:59.992 4076 3051113162712 MainThread INFO nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Creating config drive at C:\openstack\instances\instance-00000062\configdrive.iso 2021-09-10 17:05:00.030 4076 3051113162712 MainThread DEBUG oslo_concurrency.processutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000062\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_9cvmh80 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:00.343 4076 3051113162712 MainThread DEBUG oslo_concurrency.processutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000062\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_9cvmh80" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:00.726 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-00000062: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:00.766 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:00.800 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'C6CE6D5E-B6B1-4283-A722-C624618CAB1F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14707), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:00.946 4076 3051114314216 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:00.946 4076 3051114314216 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:00.946 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:00.956 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:00.956 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C6CE6D5E-B6B1-4283-A722-C624618CAB1F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115850), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:00.956 4076 3051113162712 MainThread DEBUG os_win._utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1895434856414795 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:00.984 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-c0487486-256b-42e0-8fdf-4e477d9fea83 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:00.984 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-c0487486-256b-42e0-8fdf-4e477d9fea83 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:01.017 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Successfully created port: ce3d2c5e-f487-4f16-a37b-35175bd5339e _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:05:01.050 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-c0487486-256b-42e0-8fdf-4e477d9fea83 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:01.079 4076 3051114314216 MainThread INFO nova.compute.manager [req-c0487486-256b-42e0-8fdf-4e477d9fea83 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:01.976 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '5ABF2518-FBC6-48E8-ABB2-5D8AF720BFB8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15154), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:02.109 4076 3051114314216 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:02.109 4076 3051114314216 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:02.109 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:02.109 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:02.109 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5ABF2518-FBC6-48E8-ABB2-5D8AF720BFB8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109323), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:02.109 4076 3051113162712 MainThread DEBUG os_win._utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3430097103118896 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:02.147 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-1ee3b62f-647d-48b6-95c6-b26b58b0f98b - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:02.147 4076 3051114314216 MainThread DEBUG nova.virt.hyperv.vmops [req-1ee3b62f-647d-48b6-95c6-b26b58b0f98b - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:02.194 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-1ee3b62f-647d-48b6-95c6-b26b58b0f98b - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:02.209 4076 3051114314216 MainThread INFO nova.compute.manager [req-1ee3b62f-647d-48b6-95c6-b26b58b0f98b - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:03.158 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'F8593028-67D3-49E2-809B-7C91D4C13252', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15532), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:03.266 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Successfully updated port: ce3d2c5e-f487-4f16-a37b-35175bd5339e _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:05:03.287 4076 3051113162888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:03.287 4076 3051113162888 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:03.287 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:03.287 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:03.302 4076 3051093170312 GreenThread-85 DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Acquired lock "refresh_cache-d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:03.302 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:03.302 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F8593028-67D3-49E2-809B-7C91D4C13252', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110394), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:03.302 4076 3051113162712 MainThread DEBUG os_win._utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.535693645477295 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:03.322 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-55d98ba3-2749-4e61-a41d-90b19dad76f2 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:03.322 4076 3051113162888 MainThread DEBUG nova.virt.hyperv.vmops [req-55d98ba3-2749-4e61-a41d-90b19dad76f2 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:03.366 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-55d98ba3-2749-4e61-a41d-90b19dad76f2 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:03.397 4076 3051113162888 MainThread INFO nova.compute.manager [req-55d98ba3-2749-4e61-a41d-90b19dad76f2 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:03.851 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:03.926 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Received event network-changed-ce3d2c5e-f487-4f16-a37b-35175bd5339e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:03.926 4076 3051114314216 MainThread DEBUG nova.compute.manager [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Refreshing instance network info cache due to event network-changed-ce3d2c5e-f487-4f16-a37b-35175bd5339e. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:05:04.327 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '834FCE29-D6D6-48BE-9004-5412E18EB197', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14569), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:04.473 4076 3051114312280 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:04.473 4076 3051114312280 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:04.473 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:04.473 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:04.491 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '834FCE29-D6D6-48BE-9004-5412E18EB197', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110777), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:04.491 4076 3051113162712 MainThread DEBUG os_win._utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.72534441947937 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:04.506 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-34540af2-87b6-4580-a96a-e0e9802e859c - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:04.506 4076 3051114312280 MainThread DEBUG nova.virt.hyperv.vmops [req-34540af2-87b6-4580-a96a-e0e9802e859c - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:04.551 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-34540af2-87b6-4580-a96a-e0e9802e859c - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:04.626 4076 3051114312280 MainThread INFO nova.compute.manager [req-34540af2-87b6-4580-a96a-e0e9802e859c - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:04.702 4076 3051093170312 GreenThread-85 DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Updating instance_info_cache with network_info: [{"id": "ce3d2c5e-f487-4f16-a37b-35175bd5339e", "address": "fa:16:3e:88:5f:7a", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapce3d2c5e-f4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:04.753 4076 3051093170312 GreenThread-85 DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Releasing lock "refresh_cache-d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:04.753 4076 3051093170312 GreenThread-85 DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Instance network_info: |[{"id": "ce3d2c5e-f487-4f16-a37b-35175bd5339e", "address": "fa:16:3e:88:5f:7a", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapce3d2c5e-f4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:05:04.753 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:04.753 4076 3051114314216 MainThread DEBUG nova.network.neutron [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Refreshing network info cache for port ce3d2c5e-f487-4f16-a37b-35175bd5339e _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:05:04.753 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Preparing to wait for external event network-vif-plugged-ce3d2c5e-f487-4f16-a37b-35175bd5339e prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:05:04.753 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:04.753 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:04.753 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating VM instance-00000064 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:05.514 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '45C6F1C5-B694-4507-AE9A-1DCC19B865C4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14219), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:05.655 4076 3051113162888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:05.655 4076 3051113162888 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:05.655 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:05.655 4076 3051114314392 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:05.655 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:05:05.655 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '45C6F1C5-B694-4507-AE9A-1DCC19B865C4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110466), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:05.655 4076 3051113162712 MainThread DEBUG os_win._utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.889427185058594 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:05.686 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-cbb121c5-345e-4ec2-9e30-c1f2a944e754 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:05.690 4076 3051113162888 MainThread DEBUG nova.virt.hyperv.vmops [req-cbb121c5-345e-4ec2-9e30-c1f2a944e754 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:05.736 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-cbb121c5-345e-4ec2-9e30-c1f2a944e754 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:05.749 4076 3051113162888 MainThread INFO nova.compute.manager [req-cbb121c5-345e-4ec2-9e30-c1f2a944e754 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:06.000 4076 3051114314216 MainThread DEBUG nova.network.neutron [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Updated VIF entry in instance network info cache for port ce3d2c5e-f487-4f16-a37b-35175bd5339e. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:05:06.000 4076 3051114314216 MainThread DEBUG nova.network.neutron [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Updating instance_info_cache with network_info: [{"id": "ce3d2c5e-f487-4f16-a37b-35175bd5339e", "address": "fa:16:3e:88:5f:7a", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapce3d2c5e-f4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:06.038 4076 3051114314216 MainThread DEBUG oslo_concurrency.lockutils [req-e065ebf2-4a8b-4ba3-b7cd-2b6bcbf2f18f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:06.678 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '6CBE1C27-88EB-4E79-8A56-763391427B71', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14756), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:06.819 4076 3051093170312 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:06.819 4076 3051093170312 MainThread INFO nova.compute.manager [-] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] VM Stopped (Lifecycle Event) 2021-09-10 17:05:06.819 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:06.819 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:06.819 4076 3051113162712 MainThread ERROR os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000062\' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D9342210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000062\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)instance-0000006263E43C70-B681-404D-A071-F32416D93422%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6CBE1C27-88EB-4E79-8A56-763391427B71', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112074), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\nNot enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)\n\n'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422)", 'ErrorSummaryDescription': "'instance-00000062' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000062.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:06.819 4076 3051113162712 MainThread ERROR nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-00000062 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. 2021-09-10 17:05:06.835 4076 3051113162712 MainThread INFO nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Got request to destroy instance 2021-09-10 17:05:06.835 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b2f122ec-3943-4ed7-a3ef-9f17d35a6b00 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:06.835 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [req-b2f122ec-3943-4ed7-a3ef-9f17d35a6b00 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:06.892 4076 3051093170312 MainThread DEBUG nova.compute.manager [req-b2f122ec-3943-4ed7-a3ef-9f17d35a6b00 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:06.929 4076 3051093170312 MainThread INFO nova.compute.manager [req-b2f122ec-3943-4ed7-a3ef-9f17d35a6b00 - - - - -] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:08.628 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:08.628 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:08.628 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:08.655 4076 3051113162712 MainThread DEBUG os_win.utils.compute.vmutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-00000062 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:08.670 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-00000062 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:08.718 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '3B396012-8BC9-458A-AAF2-5F8FD586028D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14828), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:08.842 4076 3051113162712 MainThread DEBUG os_win.utils.jobutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': '3B396012-8BC9-458A-AAF2-5F8FD586028D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29583), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:08.842 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:08.842 4076 3051113162712 MainThread DEBUG os_win.utils.pathutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-00000062 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:08.842 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:08.842 4076 3051113162712 MainThread ERROR nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Traceback (most recent call last): 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(*args, **kwargs) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] admin_password, network_info, block_device_info) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(self, *args, **kwds) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] should_plug_vifs=False) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'req_state': req_state}) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] time.sleep(sleep_time) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return f(*args, **kwargs) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return self._wait_for_job(job_path) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] error_desc=err_desc) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During handling of the above exception, another exception occurred: 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Traceback (most recent call last): 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] yield resources 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] accel_info=accel_info) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(*args, **kwargs) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] admin_password, network_info, block_device_info) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(self, *args, **kwds) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] should_plug_vifs=False) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'req_state': req_state}) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] time.sleep(sleep_time) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return f(*args, **kwargs) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return self._wait_for_job(job_path) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] error_desc=err_desc) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:08.842 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:08.842 4076 3051113162712 MainThread INFO nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Terminating instance 2021-09-10 17:05:08.842 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:08.842 4076 3051113162712 MainThread INFO nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Got request to destroy instance 2021-09-10 17:05:08.858 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:08.871 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:08.871 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000062" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:08.871 4076 3051113162712 MainThread INFO nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:08.871 4076 3051113162712 MainThread DEBUG nova.compute.claims [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:08.871 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:09.006 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.hostops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:09.006 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.hostops [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:09.234 4076 3051113162712 MainThread DEBUG nova.compute.provider_tree [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.scheduler.client.report [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:09.253 4076 3051113162712 MainThread ERROR nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Traceback (most recent call last): 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(*args, **kwargs) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] admin_password, network_info, block_device_info) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(self, *args, **kwds) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] should_plug_vifs=False) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'req_state': req_state}) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] time.sleep(sleep_time) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return f(*args, **kwargs) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return self._wait_for_job(job_path) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] error_desc=err_desc) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] During handling of the above exception, another exception occurred: 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Traceback (most recent call last): 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] accel_info=accel_info) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(*args, **kwargs) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] admin_password, network_info, block_device_info) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return function(self, *args, **kwds) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] should_plug_vifs=False) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'req_state': req_state}) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] time.sleep(sleep_time) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self.force_reraise() 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] raise self.value 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return f(*args, **kwargs) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] return self._wait_for_job(job_path) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] error_desc=err_desc) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. 2021-09-10 17:05:09.253 4076 3051113162712 MainThread TRACE nova.compute.manager [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.compute.utils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Build of instance 87008c7c-0917-4c78-93e1-b929cb87a160 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000062' failed to start. Not enough memory in the system to start the virtual machine instance-00000062. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000062' failed to start. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Not enough memory in the system to start the virtual machine instance-00000062 with ram size 1024 megabytes. (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) 'instance-00000062' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 63E43C70-B681-404D-A071-F32416D93422) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:05:09.253 4076 3051113162712 MainThread DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:05:13.663 4076 3051114313688 MainThread DEBUG nova.compute.manager [req-22a1271c-2698-456b-bcce-6e7fd36c865c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Received event network-vif-plugged-ce3d2c5e-f487-4f16-a37b-35175bd5339e external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:13.663 4076 3051114313688 MainThread DEBUG oslo_concurrency.lockutils [req-22a1271c-2698-456b-bcce-6e7fd36c865c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:13.663 4076 3051114313688 MainThread DEBUG oslo_concurrency.lockutils [req-22a1271c-2698-456b-bcce-6e7fd36c865c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:13.663 4076 3051114313688 MainThread DEBUG nova.compute.manager [req-22a1271c-2698-456b-bcce-6e7fd36c865c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Processing event network-vif-plugged-ce3d2c5e-f487-4f16-a37b-35175bd5339e _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:05:13.718 4076 3051113162712 MainThread DEBUG nova.network.neutron [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:13.756 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Using config drive for instance 2021-09-10 17:05:13.772 4076 3051113162712 MainThread INFO nova.compute.manager [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 87008c7c-0917-4c78-93e1-b929cb87a160] Took 4.52 seconds to deallocate network for instance. 2021-09-10 17:05:14.115 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Creating config drive at C:\openstack\instances\instance-00000064\configdrive.iso 2021-09-10 17:05:14.140 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000064\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpm85gsk6i execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:14.286 4076 3051113162712 MainThread INFO nova.scheduler.client.report [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance 87008c7c-0917-4c78-93e1-b929cb87a160 2021-09-10 17:05:14.358 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-568e6b8a-d4c0-4bb1-b65e-847f9db34869 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "87008c7c-0917-4c78-93e1-b929cb87a160" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 32.094s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:14.444 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000064\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpm85gsk6i" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:14.819 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Setting boot order for instance: instance-00000064: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:14.873 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:14.892 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'EA919ACB-116D-4578-A120-AA4BC0378743', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14748), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:15.045 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:15.045 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:15.045 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:15.049 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:15.069 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-c682c1d8-2f9a-4ab2-8b02-0b3dd0817c56 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:15.069 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-c682c1d8-2f9a-4ab2-8b02-0b3dd0817c56 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:15.108 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-c682c1d8-2f9a-4ab2-8b02-0b3dd0817c56 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:15.126 4076 3051113162712 MainThread INFO nova.compute.manager [req-c682c1d8-2f9a-4ab2-8b02-0b3dd0817c56 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:15.147 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EA919ACB-116D-4578-A120-AA4BC0378743', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112678), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:15.147 4076 3051113161128 MainThread DEBUG os_win._utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.27330875396728516 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:16.179 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '64F51DF7-E9F5-452E-AD8B-5278F370AB0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15180), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:16.297 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:16.297 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:16.297 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:16.297 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:16.317 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '64F51DF7-E9F5-452E-AD8B-5278F370AB0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111396), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:16.317 4076 3051113161128 MainThread DEBUG os_win._utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4434893131256104 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:16.331 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-f368d7ce-1156-4c6e-b5c0-0d6ba2399c6f - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:16.331 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-f368d7ce-1156-4c6e-b5c0-0d6ba2399c6f - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:16.374 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-f368d7ce-1156-4c6e-b5c0-0d6ba2399c6f - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:16.428 4076 3051113162712 MainThread INFO nova.compute.manager [req-f368d7ce-1156-4c6e-b5c0-0d6ba2399c6f - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:17.362 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '98BCA7D1-0D5B-4CEB-B76D-8275C86B3D5D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17579), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:17.485 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:17.485 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:17.485 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:17.485 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:17.503 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '98BCA7D1-0D5B-4CEB-B76D-8275C86B3D5D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113609), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:17.503 4076 3051113161128 MainThread DEBUG os_win._utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.629556179046631 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:17.522 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-00fab67c-9da0-4fc9-b1cb-7b2a74dcd854 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:17.522 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-00fab67c-9da0-4fc9-b1cb-7b2a74dcd854 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:17.562 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-00fab67c-9da0-4fc9-b1cb-7b2a74dcd854 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:17.609 4076 3051113162712 MainThread INFO nova.compute.manager [req-00fab67c-9da0-4fc9-b1cb-7b2a74dcd854 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:18.524 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'CDCEE84E-D162-4D97-ADDD-C25071400A5E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:18.663 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:18.663 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:18.663 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:18.663 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:18.672 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CDCEE84E-D162-4D97-ADDD-C25071400A5E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109803), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:18.672 4076 3051113161128 MainThread DEBUG os_win._utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.798733711242676 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:18.693 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-557aa787-4262-4873-a181-ad74fa5e4840 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:18.693 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-557aa787-4262-4873-a181-ad74fa5e4840 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:18.740 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-557aa787-4262-4873-a181-ad74fa5e4840 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:18.788 4076 3051113162712 MainThread INFO nova.compute.manager [req-557aa787-4262-4873-a181-ad74fa5e4840 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:19.201 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:19.241 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:19.475 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:19.475 4076 3051114314568 MainThread DEBUG nova.virt.hardware [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:19.491 4076 3051114314568 MainThread INFO nova.compute.claims [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Claim successful on node n-h1-808474-1 2021-09-10 17:05:19.639 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:19.639 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:19.710 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '26E9B742-DF00-4971-BF1F-37E091B9A863', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16200), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:19.767 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "94d2efe1-0d5a-49bf-ba29-3c312237f77d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:19.792 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:19.847 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:19.847 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:19.847 4076 3051114314040 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:19.847 4076 3051114314040 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:19.858 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '26E9B742-DF00-4971-BF1F-37E091B9A863', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112839), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:19.858 4076 3051113161128 MainThread DEBUG os_win._utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.984592437744141 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:19.874 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-0ad0a44d-fecb-484d-b186-f8f33aaf24e8 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:19.874 4076 3051114314040 MainThread DEBUG nova.virt.hyperv.vmops [req-0ad0a44d-fecb-484d-b186-f8f33aaf24e8 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:19.874 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:19.920 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-0ad0a44d-fecb-484d-b186-f8f33aaf24e8 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:19.951 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:19.951 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:19.951 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:19.951 4076 3051114314040 MainThread INFO nova.compute.manager [req-0ad0a44d-fecb-484d-b186-f8f33aaf24e8 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:19.995 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:19.995 4076 3051114314744 MainThread DEBUG nova.virt.hardware [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:19.995 4076 3051114314744 MainThread INFO nova.compute.claims [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Claim successful on node n-h1-808474-1 2021-09-10 17:05:20.046 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:20.046 4076 3051114314920 GreenThread-86 DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:05:20.061 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:05:20.296 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.hostops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:20.314 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.hostops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:20.438 4076 3051114314920 GreenThread-86 DEBUG nova.policy [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:05:20.486 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:05:20.486 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Spawning new instance 2021-09-10 17:05:20.513 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:20.513 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:20.513 4076 3051114314568 MainThread DEBUG nova.block_device [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:05:20.513 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:20.513 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:20.524 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-00000063 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:05:20.524 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000063\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:05:20.541 4076 3051114314744 MainThread DEBUG nova.compute.provider_tree [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:20.545 4076 3051114314744 MainThread DEBUG nova.scheduler.client.report [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:20.545 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:20.545 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:20.545 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Resizing VHD C:\openstack\instances\instance-00000063\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:05:20.658 4076 3051114314744 MainThread DEBUG nova.compute.utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:05:20.658 4076 3051114315096 GreenThread-87 DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not allocating networking since 'none' was specified. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1752 2021-09-10 17:05:20.711 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:20.890 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '39B3C9C6-1FCC-48C8-B551-18C7DE5D0206', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15985), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:20.958 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:05:20.968 4076 3051114314744 MainThread INFO nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Spawning new instance 2021-09-10 17:05:20.968 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:20.968 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:20.968 4076 3051114314744 MainThread DEBUG nova.block_device [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:05:20.986 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:20.986 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:20.986 4076 3051114314744 MainThread DEBUG os_win.utils.pathutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Creating directory: C:\openstack\instances\instance-00000065 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:05:20.986 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000065\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:05:21.031 4076 3051114315272 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:21.031 4076 3051114315272 MainThread INFO nova.compute.manager [-] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:21.031 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:21.031 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:21.047 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Resizing VHD C:\openstack\instances\instance-00000065\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:05:21.047 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000064\' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB6925410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000064\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)instance-00000064336523A8-F235-4F1F-B54B-FF4D8FB69254%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '39B3C9C6-1FCC-48C8-B551-18C7DE5D0206', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110550), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\nNot enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)\n\n'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254)", 'ErrorSummaryDescription': "'instance-00000064' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000064.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:21.061 4076 3051113161128 MainThread ERROR nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Failed to change vm state of instance-00000064 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. 2021-09-10 17:05:21.061 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Got request to destroy instance 2021-09-10 17:05:21.061 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-1950cfb1-5d69-4c4d-b2ef-f58204b132b3 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:21.061 4076 3051114315272 MainThread DEBUG nova.virt.hyperv.vmops [req-1950cfb1-5d69-4c4d-b2ef-f58204b132b3 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:21.061 4076 3051114314744 MainThread DEBUG os_win.utils.compute.vmutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Creating VM instance-00000065 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:21.109 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-1950cfb1-5d69-4c4d-b2ef-f58204b132b3 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:21.145 4076 3051114315272 MainThread INFO nova.compute.manager [req-1950cfb1-5d69-4c4d-b2ef-f58204b132b3 - - - - -] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:21.227 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:21.249 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:21.429 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:21.429 4076 3051114315096 MainThread DEBUG nova.virt.hardware [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:21.429 4076 3051114315096 MainThread INFO nova.compute.claims [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Claim successful on node n-h1-808474-1 2021-09-10 17:05:21.561 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.hostops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:21.584 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.hostops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:21.798 4076 3051114315096 MainThread DEBUG nova.compute.provider_tree [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:21.826 4076 3051114315096 MainThread DEBUG nova.scheduler.client.report [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:21.826 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:21.826 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:21.945 4076 3051114315096 MainThread DEBUG nova.compute.utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:05:21.945 4076 3051114315272 GreenThread-88 DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not allocating networking since 'none' was specified. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1752 2021-09-10 17:05:21.967 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:22.143 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:05:22.143 4076 3051114315096 MainThread INFO nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Spawning new instance 2021-09-10 17:05:22.166 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG nova.block_device [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG os_win.utils.pathutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Creating directory: C:\openstack\instances\instance-00000066 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:05:22.170 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000066\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:05:22.203 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Resizing VHD C:\openstack\instances\instance-00000066\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:05:22.222 4076 3051114315096 MainThread DEBUG os_win.utils.compute.vmutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Creating VM instance-00000066 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:22.269 4076 3051114314744 MainThread INFO nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Using config drive for instance 2021-09-10 17:05:22.470 4076 3051114314744 MainThread INFO nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Creating config drive at C:\openstack\instances\instance-00000065\configdrive.iso 2021-09-10 17:05:22.501 4076 3051114314744 MainThread DEBUG oslo_concurrency.processutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000065\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl193jiog execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:22.798 4076 3051114314744 MainThread DEBUG oslo_concurrency.processutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000065\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl193jiog" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:23.031 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Successfully created port: 0df9a276-30c3-45e9-9e79-d1ea75886aa5 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:05:23.237 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Setting boot order for instance: instance-00000065: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:23.237 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:23.237 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:23.237 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:23.286 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed vm state of instance-00000064 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:23.286 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed state of VM instance-00000064 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:23.300 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:23.342 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'DB507693-6F6D-406A-9132-872AE7F516EE', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16485), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:23.342 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': 'EA25F7E9-C0A8-4117-871F-846D79A69449', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16678), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:23.468 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:23.468 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:23.468 4076 3051114315624 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:23.468 4076 3051114315624 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job succeeded: {'InstanceID': 'DB507693-6F6D-406A-9132-872AE7F516EE', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32776), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:23.468 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:23.468 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Removing directory: C:\openstack\instances\instance-00000064 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:23.468 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:23.468 4076 3051113161128 MainThread ERROR nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Traceback (most recent call last): 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(*args, **kwargs) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] admin_password, network_info, block_device_info) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(self, *args, **kwds) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] should_plug_vifs=False) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'req_state': req_state}) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] time.sleep(sleep_time) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return f(*args, **kwargs) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return self._wait_for_job(job_path) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] error_desc=err_desc) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During handling of the above exception, another exception occurred: 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Traceback (most recent call last): 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] yield resources 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] accel_info=accel_info) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(*args, **kwargs) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] admin_password, network_info, block_device_info) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(self, *args, **kwds) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] should_plug_vifs=False) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'req_state': req_state}) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] time.sleep(sleep_time) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return f(*args, **kwargs) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return self._wait_for_job(job_path) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] error_desc=err_desc) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:23.468 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:23.468 4076 3051113161128 MainThread INFO nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Terminating instance 2021-09-10 17:05:23.468 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:23.468 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Got request to destroy instance 2021-09-10 17:05:23.490 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EA25F7E9-C0A8-4117-871F-846D79A69449', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=94196), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:23.490 4076 3051114314744 MainThread DEBUG os_win._utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17023324966430664 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:23.490 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:23.490 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:23.490 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-00000064" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:23.490 4076 3051113161128 MainThread INFO nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:23.490 4076 3051113161128 MainThread DEBUG nova.compute.claims [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:23.509 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:23.515 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-1442a760-ca4a-4648-9ee4-3d25a4be68dc - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:23.515 4076 3051114315624 MainThread DEBUG nova.virt.hyperv.vmops [req-1442a760-ca4a-4648-9ee4-3d25a4be68dc - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:23.545 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-1442a760-ca4a-4648-9ee4-3d25a4be68dc - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:23.634 4076 3051114315624 MainThread INFO nova.compute.manager [req-1442a760-ca4a-4648-9ee4-3d25a4be68dc - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:23.672 4076 3051114315096 MainThread INFO nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Using config drive for instance 2021-09-10 17:05:23.717 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:23.717 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:23.858 4076 3051114315096 MainThread INFO nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Creating config drive at C:\openstack\instances\instance-00000066\configdrive.iso 2021-09-10 17:05:23.898 4076 3051114315096 MainThread DEBUG oslo_concurrency.processutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000066\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmich_obr execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:24.103 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:24.126 4076 3051113161128 MainThread ERROR nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Traceback (most recent call last): 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(*args, **kwargs) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] admin_password, network_info, block_device_info) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(self, *args, **kwds) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] should_plug_vifs=False) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'req_state': req_state}) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] time.sleep(sleep_time) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return f(*args, **kwargs) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return self._wait_for_job(job_path) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] error_desc=err_desc) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] During handling of the above exception, another exception occurred: 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Traceback (most recent call last): 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] accel_info=accel_info) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(*args, **kwargs) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] admin_password, network_info, block_device_info) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return function(self, *args, **kwds) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] should_plug_vifs=False) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'req_state': req_state}) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] time.sleep(sleep_time) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self.force_reraise() 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] raise self.value 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return f(*args, **kwargs) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] return self._wait_for_job(job_path) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] error_desc=err_desc) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. 2021-09-10 17:05:24.126 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Build of instance d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000064' failed to start. Not enough memory in the system to start the virtual machine instance-00000064. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000064' failed to start. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Not enough memory in the system to start the virtual machine instance-00000064 with ram size 1024 megabytes. (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) 'instance-00000064' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 336523A8-F235-4F1F-B54B-FF4D8FB69254) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:05:24.126 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:05:24.172 4076 3051114315096 MainThread DEBUG oslo_concurrency.processutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000066\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmich_obr" returned: 0 in 0.266s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:24.504 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '7C91A1FC-A200-4321-8B07-B480F75FD896', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14885), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:24.586 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Setting boot order for instance: instance-00000066: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:24.615 4076 3051114315624 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:24.615 4076 3051114315624 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:24.615 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:24.615 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:24.642 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7C91A1FC-A200-4321-8B07-B480F75FD896', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=91449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:24.642 4076 3051114314744 MainThread DEBUG os_win._utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.322730302810669 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:24.642 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:24.642 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-2245fdc2-0a94-45cf-b04b-f15cf05d7cdf - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:24.642 4076 3051114315624 MainThread DEBUG nova.virt.hyperv.vmops [req-2245fdc2-0a94-45cf-b04b-f15cf05d7cdf - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:24.656 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': 'D05865F0-CB3F-4B71-B383-E7BB8B86B204', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14968), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:24.696 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-2245fdc2-0a94-45cf-b04b-f15cf05d7cdf - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:24.718 4076 3051114315624 MainThread INFO nova.compute.manager [req-2245fdc2-0a94-45cf-b04b-f15cf05d7cdf - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:24.789 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:24.789 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:24.789 4076 3051114315624 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:24.789 4076 3051114315624 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:24.799 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D05865F0-CB3F-4B71-B383-E7BB8B86B204', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=97921), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:24.799 4076 3051114315096 MainThread DEBUG os_win._utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.15632414817810059 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:24.822 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-7b6b6e3f-fc1a-4d7c-a824-13613da5b2b6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:24.822 4076 3051114315624 MainThread DEBUG nova.virt.hyperv.vmops [req-7b6b6e3f-fc1a-4d7c-a824-13613da5b2b6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:24.865 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-7b6b6e3f-fc1a-4d7c-a824-13613da5b2b6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:24.920 4076 3051114315624 MainThread INFO nova.compute.manager [req-7b6b6e3f-fc1a-4d7c-a824-13613da5b2b6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:25.165 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Successfully updated port: 0df9a276-30c3-45e9-9e79-d1ea75886aa5 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:05:25.195 4076 3051114314920 GreenThread-86 DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:25.195 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:25.243 4076 3051093460072 MainThread DEBUG nova.compute.manager [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Received event network-changed-0df9a276-30c3-45e9-9e79-d1ea75886aa5 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:25.264 4076 3051093460072 MainThread DEBUG nova.compute.manager [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Refreshing instance network info cache due to event network-changed-0df9a276-30c3-45e9-9e79-d1ea75886aa5. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:05:25.409 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:25.686 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '6711E584-0733-4A0C-A865-D02BFD58214D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15279), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:25.787 4076 3051114315624 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:25.787 4076 3051114315624 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:25.787 4076 3051114314040 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:25.787 4076 3051114314040 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:25.827 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-2d23476a-5107-4734-b8a7-3c315ba6e84c - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:25.827 4076 3051114314040 MainThread DEBUG nova.virt.hyperv.vmops [req-2d23476a-5107-4734-b8a7-3c315ba6e84c - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:25.827 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '7CC92DD8-5496-47CD-9DA9-5B4DF206DF84', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:25.844 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6711E584-0733-4A0C-A865-D02BFD58214D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=94189), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:25.844 4076 3051114314744 MainThread DEBUG os_win._utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5240695476531982 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:25.869 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-2d23476a-5107-4734-b8a7-3c315ba6e84c - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:25.903 4076 3051114314040 MainThread INFO nova.compute.manager [req-2d23476a-5107-4734-b8a7-3c315ba6e84c - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:25.937 4076 3051114314040 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:25.937 4076 3051114314040 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:25.937 4076 3051114315624 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:25.937 4076 3051114315624 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:25.970 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-93b2b7e2-96e2-4104-8464-7e785f1cedb6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:25.970 4076 3051114314040 MainThread DEBUG nova.virt.hyperv.vmops [req-93b2b7e2-96e2-4104-8464-7e785f1cedb6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:25.970 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7CC92DD8-5496-47CD-9DA9-5B4DF206DF84', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=91914), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:25.970 4076 3051114315096 MainThread DEBUG os_win._utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.327660083770752 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:25.999 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-93b2b7e2-96e2-4104-8464-7e785f1cedb6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:26.031 4076 3051114314040 MainThread INFO nova.compute.manager [req-93b2b7e2-96e2-4104-8464-7e785f1cedb6 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:26.255 4076 3051114314920 GreenThread-86 DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Updating instance_info_cache with network_info: [{"id": "0df9a276-30c3-45e9-9e79-d1ea75886aa5", "address": "fa:16:3e:4f:c0:2b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0df9a276-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:26.274 4076 3051114314920 GreenThread-86 DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:26.274 4076 3051114314920 GreenThread-86 DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Instance network_info: |[{"id": "0df9a276-30c3-45e9-9e79-d1ea75886aa5", "address": "fa:16:3e:4f:c0:2b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0df9a276-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:05:26.274 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Preparing to wait for external event network-vif-plugged-0df9a276-30c3-45e9-9e79-d1ea75886aa5 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:05:26.274 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:26.274 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:26.274 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-00000063 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:26.274 4076 3051093460072 MainThread DEBUG oslo_concurrency.lockutils [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:26.274 4076 3051093460072 MainThread DEBUG nova.network.neutron [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Refreshing network info cache for port 0df9a276-30c3-45e9-9e79-d1ea75886aa5 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:05:26.768 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:26.783 4076 3051113161128 MainThread INFO nova.compute.manager [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d] Took 2.66 seconds to deallocate network for instance. 2021-09-10 17:05:26.877 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '57947D75-246F-493E-9A7E-82CDA47A5498', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16697), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:27.001 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:27.001 4076 3051114315448 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:27.001 4076 3051114315272 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:27.001 4076 3051114315272 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:27.001 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '03A73407-9716-48E8-9A80-86805E85C57B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16465), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:27.025 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '57947D75-246F-493E-9A7E-82CDA47A5498', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=99102), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:27.029 4076 3051114314744 MainThread DEBUG os_win._utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7098472118377686 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:27.029 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-7bdf7707-597d-4f6a-9b2e-6b30a718c9e3 - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:27.029 4076 3051114315272 MainThread DEBUG nova.virt.hyperv.vmops [req-7bdf7707-597d-4f6a-9b2e-6b30a718c9e3 - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:27.083 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-7bdf7707-597d-4f6a-9b2e-6b30a718c9e3 - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:27.120 4076 3051093460248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:27.120 4076 3051093460248 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:27.120 4076 3051114315624 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:27.120 4076 3051114315624 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:27.132 4076 3051114315272 MainThread INFO nova.compute.manager [req-7bdf7707-597d-4f6a-9b2e-6b30a718c9e3 - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:27.147 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-3ded6a53-9a44-4e7a-8265-e051d68d0877 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:27.147 4076 3051114315624 MainThread DEBUG nova.virt.hyperv.vmops [req-3ded6a53-9a44-4e7a-8265-e051d68d0877 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:27.157 4076 3051113161128 MainThread INFO nova.scheduler.client.report [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Deleted allocations for instance d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d 2021-09-10 17:05:27.157 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '03A73407-9716-48E8-9A80-86805E85C57B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=92520), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:27.174 4076 3051114315096 MainThread DEBUG os_win._utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.531519889831543 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:27.207 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-338276e3-d960-4845-be9d-42db39d9cc04 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "d2485dc8-cd44-42ec-87cb-0a2cfbe0de7d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.343s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:27.207 4076 3051114315624 MainThread DEBUG nova.compute.manager [req-3ded6a53-9a44-4e7a-8265-e051d68d0877 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:27.219 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:05:27.264 4076 3051114315624 MainThread INFO nova.compute.manager [req-3ded6a53-9a44-4e7a-8265-e051d68d0877 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:28.068 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '56F6A576-62FB-4765-A015-0E647EDA1EFD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20224), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:28.100 4076 3051093460072 MainThread DEBUG nova.network.neutron [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Updated VIF entry in instance network info cache for port 0df9a276-30c3-45e9-9e79-d1ea75886aa5. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:05:28.100 4076 3051093460072 MainThread DEBUG nova.network.neutron [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Updating instance_info_cache with network_info: [{"id": "0df9a276-30c3-45e9-9e79-d1ea75886aa5", "address": "fa:16:3e:4f:c0:2b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0df9a276-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:28.112 4076 3051093460072 MainThread DEBUG oslo_concurrency.lockutils [req-05797135-a742-4726-b218-cd472af7171f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:28.172 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:28.172 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:28.172 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:28.172 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:28.187 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': 'DE1E6511-AF02-41E7-B691-53B4D67D1188', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18159), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:28.187 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '56F6A576-62FB-4765-A015-0E647EDA1EFD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=92764), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:28.205 4076 3051114314744 MainThread DEBUG os_win._utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.885900259017944 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:28.205 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-71291ce3-2584-42c3-b12f-767db3ff888f - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:28.205 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-71291ce3-2584-42c3-b12f-767db3ff888f - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:28.236 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-71291ce3-2584-42c3-b12f-767db3ff888f - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:28.274 4076 3051113161128 MainThread INFO nova.compute.manager [req-71291ce3-2584-42c3-b12f-767db3ff888f - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:28.297 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:28.297 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:28.297 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:28.297 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:28.326 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-47e08eba-bd5b-499b-b8d6-192ecc327c31 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:28.326 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-47e08eba-bd5b-499b-b8d6-192ecc327c31 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:28.339 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DE1E6511-AF02-41E7-B691-53B4D67D1188', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=93634), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:28.339 4076 3051114315096 MainThread DEBUG os_win._utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6969239711761475 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:28.384 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-47e08eba-bd5b-499b-b8d6-192ecc327c31 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:28.406 4076 3051113161128 MainThread INFO nova.compute.manager [req-47e08eba-bd5b-499b-b8d6-192ecc327c31 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:29.234 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': 'C243FAC3-9FAD-40CF-BAC3-ECF777160A73', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15533), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:29.340 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:29.340 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:29.340 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:29.340 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] VM Stopped (Lifecycle Event) 2021-09-10 17:05:29.362 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '594C508C-3282-449A-B4A4-4EDC0ED27632', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18505), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:29.362 4076 3051114314744 MainThread ERROR os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000065\' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000065\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)instance-0000006544124959-9A31-45C7-94DC-69D16C42F28F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C243FAC3-9FAD-40CF-BAC3-ECF777160A73', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=92338), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\nNot enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)\n\n'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F)", 'ErrorSummaryDescription': "'instance-00000065' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000065.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:29.362 4076 3051114314744 MainThread ERROR nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Failed to change vm state of instance-00000065 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. 2021-09-10 17:05:29.362 4076 3051114314744 MainThread INFO nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Got request to destroy instance 2021-09-10 17:05:29.392 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-7d68cf9a-966d-4559-a887-d1795d6ee83e - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:29.392 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-7d68cf9a-966d-4559-a887-d1795d6ee83e - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:29.439 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-7d68cf9a-966d-4559-a887-d1795d6ee83e - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:29.470 4076 3051114314040 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:29.470 4076 3051114314040 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:29.494 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:29.494 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:29.494 4076 3051113161128 MainThread INFO nova.compute.manager [req-7d68cf9a-966d-4559-a887-d1795d6ee83e - - - - -] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:29.504 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '594C508C-3282-449A-B4A4-4EDC0ED27632', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=94066), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:29.504 4076 3051114315096 MainThread DEBUG os_win._utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.861331939697266 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:29.504 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-fcff9df3-c9d4-44c4-985e-14c7a6fd9d9f - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:29.520 4076 3051114314040 MainThread DEBUG nova.virt.hyperv.vmops [req-fcff9df3-c9d4-44c4-985e-14c7a6fd9d9f - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:29.561 4076 3051114314040 MainThread DEBUG nova.compute.manager [req-fcff9df3-c9d4-44c4-985e-14c7a6fd9d9f - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:29.592 4076 3051114314040 MainThread INFO nova.compute.manager [req-fcff9df3-c9d4-44c4-985e-14c7a6fd9d9f - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:30.533 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': 'B336463F-1F38-4031-855D-37EF8625F0DE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16161), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:30.642 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:30.642 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:30.642 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:30.642 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] VM Stopped (Lifecycle Event) 2021-09-10 17:05:30.674 4076 3051114315096 MainThread ERROR os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000066\' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000066\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)instance-00000066D737F854-64D7-4FAC-BFCE-17EDF22C6FE7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B336463F-1F38-4031-855D-37EF8625F0DE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=95095), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\nNot enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)\n\n'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7)", 'ErrorSummaryDescription': "'instance-00000066' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000066.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:30.674 4076 3051114315096 MainThread ERROR nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Failed to change vm state of instance-00000066 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. 2021-09-10 17:05:30.674 4076 3051114315096 MainThread INFO nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Got request to destroy instance 2021-09-10 17:05:30.688 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-edfbe35d-8050-48a0-bb85-96c58635b551 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:30.688 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-edfbe35d-8050-48a0-bb85-96c58635b551 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:30.737 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:30.737 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-edfbe35d-8050-48a0-bb85-96c58635b551 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:30.774 4076 3051113161128 MainThread INFO nova.compute.manager [req-edfbe35d-8050-48a0-bb85-96c58635b551 - - - - -] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:30.774 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:30.941 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:30.941 4076 3051114314920 MainThread DEBUG nova.virt.hardware [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:30.941 4076 3051114314920 MainThread INFO nova.compute.claims [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Claim successful on node n-h1-808474-1 2021-09-10 17:05:31.143 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.hostops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:31.160 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.hostops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:31.160 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:31.160 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:31.160 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.183 4076 3051114314744 MainThread DEBUG os_win.utils.compute.vmutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Successfully changed vm state of instance-00000065 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:31.183 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Successfully changed state of VM instance-00000065 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:31.236 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '9E489893-F263-43F9-A73D-3E59BAB8361C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15831), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:31.272 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-0b03c8c8-0a92-4610-b7e1-460dbd652a06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Received event network-vif-plugged-0df9a276-30c3-45e9-9e79-d1ea75886aa5 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:31.272 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-0b03c8c8-0a92-4610-b7e1-460dbd652a06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:31.272 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-0b03c8c8-0a92-4610-b7e1-460dbd652a06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.272 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-0b03c8c8-0a92-4610-b7e1-460dbd652a06 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Processing event network-vif-plugged-0df9a276-30c3-45e9-9e79-d1ea75886aa5 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:05:31.318 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Using config drive for instance 2021-09-10 17:05:31.358 4076 3051114314744 MainThread DEBUG os_win.utils.jobutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job succeeded: {'InstanceID': '9E489893-F263-43F9-A73D-3E59BAB8361C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31501), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:31.358 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:31.358 4076 3051114314744 MainThread DEBUG os_win.utils.pathutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Removing directory: C:\openstack\instances\instance-00000065 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:31.358 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.358 4076 3051114314744 MainThread ERROR nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Traceback (most recent call last): 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(*args, **kwargs) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] admin_password, network_info, block_device_info) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(self, *args, **kwds) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] should_plug_vifs=False) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'req_state': req_state}) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] time.sleep(sleep_time) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return f(*args, **kwargs) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return self._wait_for_job(job_path) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] error_desc=err_desc) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During handling of the above exception, another exception occurred: 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Traceback (most recent call last): 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] yield resources 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] accel_info=accel_info) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(*args, **kwargs) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] admin_password, network_info, block_device_info) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(self, *args, **kwds) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] should_plug_vifs=False) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'req_state': req_state}) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] time.sleep(sleep_time) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return f(*args, **kwargs) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return self._wait_for_job(job_path) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] error_desc=err_desc) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:31.358 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:31.389 4076 3051114314744 MainThread INFO nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Terminating instance 2021-09-10 17:05:31.389 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Acquired lock "refresh_cache-94d2efe1-0d5a-49bf-ba29-3c312237f77d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:31.389 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:31.396 4076 3051114314920 MainThread DEBUG nova.compute.provider_tree [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:31.410 4076 3051114314920 MainThread DEBUG nova.scheduler.client.report [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:31.410 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.410 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:31.514 4076 3051114314920 MainThread DEBUG nova.compute.utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:05:31.514 4076 3051114315272 GreenThread-89 DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:05:31.529 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:05:31.548 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:31.634 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:31.660 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Creating config drive at C:\openstack\instances\instance-00000063\configdrive.iso 2021-09-10 17:05:31.701 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000063\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp5l5z20b4 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:31.826 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:05:31.850 4076 3051114314920 MainThread INFO nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Spawning new instance 2021-09-10 17:05:31.863 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:31.863 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.863 4076 3051114314920 MainThread DEBUG nova.block_device [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:05:31.863 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:31.875 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:31.875 4076 3051114314920 MainThread DEBUG os_win.utils.pathutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Creating directory: C:\openstack\instances\instance-00000067 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:05:31.875 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000067\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:05:31.890 4076 3051114315272 GreenThread-89 DEBUG nova.policy [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5e00875ca35f4501afe5474947cd0708', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89ea24a7b47a4f4a9015423bfcc23f3b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:05:31.905 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Resizing VHD C:\openstack\instances\instance-00000067\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:05:32.009 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000063\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp5l5z20b4" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:32.275 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:32.300 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Releasing lock "refresh_cache-94d2efe1-0d5a-49bf-ba29-3c312237f77d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:32.300 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:32.300 4076 3051114314744 MainThread INFO nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Got request to destroy instance 2021-09-10 17:05:32.300 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.vmops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:32.321 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:32.321 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000065" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:32.321 4076 3051114314744 MainThread INFO nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:32.321 4076 3051114314744 MainThread DEBUG oslo.service.loopingcall [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.._deallocate_network_with_retries to return. func c:\python37\lib\site-packages\oslo_service\loopingcall.py:435 2021-09-10 17:05:32.321 4076 3051114315448 GreenThread-90 DEBUG nova.compute.manager [-] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:05:32.321 4076 3051114314744 MainThread DEBUG nova.compute.claims [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:32.321 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:32.415 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-00000063: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:32.467 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:32.488 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.hostops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:32.504 4076 3051114314744 MainThread DEBUG nova.virt.hyperv.hostops [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:32.516 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '173D3161-6BE7-4057-8D6E-D183A4F93148', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16424), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:32.639 4076 3051114315448 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:32.639 4076 3051114315448 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:32.639 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:32.663 4076 3051114314040 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:32.663 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '173D3161-6BE7-4057-8D6E-D183A4F93148', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117357), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:32.663 4076 3051114314568 MainThread DEBUG os_win._utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19612789154052734 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:32.675 4076 3051114315448 MainThread DEBUG nova.compute.manager [req-56375d4b-ef5b-460d-bae8-87f6a901a370 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:32.691 4076 3051114315448 MainThread DEBUG nova.virt.hyperv.vmops [req-56375d4b-ef5b-460d-bae8-87f6a901a370 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:32.730 4076 3051114314744 MainThread DEBUG nova.compute.provider_tree [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:32.744 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:32.744 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:32.744 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:32.753 4076 3051114315448 MainThread DEBUG nova.compute.manager [req-56375d4b-ef5b-460d-bae8-87f6a901a370 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:32.753 4076 3051114314744 MainThread DEBUG nova.scheduler.client.report [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:32.753 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:32.753 4076 3051114314744 MainThread ERROR nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Traceback (most recent call last): 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(*args, **kwargs) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] admin_password, network_info, block_device_info) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(self, *args, **kwds) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] should_plug_vifs=False) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'req_state': req_state}) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] time.sleep(sleep_time) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return f(*args, **kwargs) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return self._wait_for_job(job_path) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] error_desc=err_desc) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] During handling of the above exception, another exception occurred: 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Traceback (most recent call last): 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] accel_info=accel_info) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(*args, **kwargs) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] admin_password, network_info, block_device_info) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return function(self, *args, **kwds) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] should_plug_vifs=False) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'req_state': req_state}) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] time.sleep(sleep_time) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self.force_reraise() 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] raise self.value 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return f(*args, **kwargs) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] return self._wait_for_job(job_path) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] error_desc=err_desc) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. 2021-09-10 17:05:32.753 4076 3051114314744 MainThread TRACE nova.compute.manager [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] 2021-09-10 17:05:32.753 4076 3051114314744 MainThread DEBUG nova.compute.utils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:32.766 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Build of instance 94d2efe1-0d5a-49bf-ba29-3c312237f77d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000065' failed to start. Not enough memory in the system to start the virtual machine instance-00000065. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000065' failed to start. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Not enough memory in the system to start the virtual machine instance-00000065 with ram size 1024 megabytes. (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) 'instance-00000065' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 44124959-9A31-45C7-94DC-69D16C42F28F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:32.766 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:32.766 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Acquired lock "refresh_cache-94d2efe1-0d5a-49bf-ba29-3c312237f77d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:32.766 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:32.766 4076 3051114315096 MainThread DEBUG os_win.utils.compute.vmutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Successfully changed vm state of instance-00000066 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:32.766 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Successfully changed state of VM instance-00000066 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:32.788 4076 3051114315448 MainThread INFO nova.compute.manager [req-56375d4b-ef5b-460d-bae8-87f6a901a370 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:32.835 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for WMI job: {'InstanceID': '6AD76254-B67F-4C64-9ED6-B66622BC5D58', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13991), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:32.844 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG os_win.utils.jobutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] WMI job succeeded: {'InstanceID': '6AD76254-B67F-4C64-9ED6-B66622BC5D58', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=27828), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG os_win.utils.pathutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Removing directory: C:\openstack\instances\instance-00000066 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:32.960 4076 3051114315096 MainThread ERROR nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Traceback (most recent call last): 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(*args, **kwargs) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] admin_password, network_info, block_device_info) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(self, *args, **kwds) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] should_plug_vifs=False) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'req_state': req_state}) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] time.sleep(sleep_time) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return f(*args, **kwargs) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return self._wait_for_job(job_path) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] error_desc=err_desc) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During handling of the above exception, another exception occurred: 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Traceback (most recent call last): 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] yield resources 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] accel_info=accel_info) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(*args, **kwargs) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] admin_password, network_info, block_device_info) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(self, *args, **kwds) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] should_plug_vifs=False) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'req_state': req_state}) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] time.sleep(sleep_time) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return f(*args, **kwargs) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return self._wait_for_job(job_path) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] error_desc=err_desc) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:32.960 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:32.960 4076 3051114315096 MainThread INFO nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Terminating instance 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Acquired lock "refresh_cache-d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:32.960 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:33.139 4076 3051114314744 MainThread DEBUG nova.network.neutron [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:33.139 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Releasing lock "refresh_cache-94d2efe1-0d5a-49bf-ba29-3c312237f77d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:33.139 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:33.139 4076 3051114314744 MainThread DEBUG nova.compute.manager [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: 94d2efe1-0d5a-49bf-ba29-3c312237f77d] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:05:33.173 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:33.486 4076 3051114314744 MainThread INFO nova.scheduler.client.report [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Deleted allocations for instance 94d2efe1-0d5a-49bf-ba29-3c312237f77d 2021-09-10 17:05:33.567 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [req-0a0e5936-e6a2-4bd5-b010-7960f41d2c13 ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "94d2efe1-0d5a-49bf-ba29-3c312237f77d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 13.797s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:33.677 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:33.702 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'F9DC4F4F-0908-4099-825D-C57B3CE26120', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15257), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:33.749 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Releasing lock "refresh_cache-d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:33.749 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:33.749 4076 3051114315096 MainThread INFO nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Got request to destroy instance 2021-09-10 17:05:33.766 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:33.766 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:33.766 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "instance-00000066" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:33.766 4076 3051114315096 MainThread INFO nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:33.766 4076 3051114315096 MainThread DEBUG oslo.service.loopingcall [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.._deallocate_network_with_retries to return. func c:\python37\lib\site-packages\oslo_service\loopingcall.py:435 2021-09-10 17:05:33.766 4076 3051114315448 GreenThread-91 DEBUG nova.compute.manager [-] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:05:33.766 4076 3051114315096 MainThread DEBUG nova.compute.claims [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:33.785 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:33.829 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:33.829 4076 3051114314744 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:33.829 4076 3051114315448 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:33.829 4076 3051114315448 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:33.829 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9DC4F4F-0908-4099-825D-C57B3CE26120', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110827), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:33.829 4076 3051114314568 MainThread DEBUG os_win._utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.362344741821289 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:33.889 4076 3051114315448 MainThread DEBUG nova.compute.manager [req-4cee3f6a-04ea-42da-96f7-4ff56e70fbd4 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:33.889 4076 3051114315448 MainThread DEBUG nova.virt.hyperv.vmops [req-4cee3f6a-04ea-42da-96f7-4ff56e70fbd4 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:33.924 4076 3051114315448 MainThread DEBUG nova.compute.manager [req-4cee3f6a-04ea-42da-96f7-4ff56e70fbd4 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:33.982 4076 3051114315448 MainThread INFO nova.compute.manager [req-4cee3f6a-04ea-42da-96f7-4ff56e70fbd4 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:34.011 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.hostops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:34.017 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.hostops [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:34.227 4076 3051114315096 MainThread DEBUG nova.compute.provider_tree [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG nova.scheduler.client.report [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.454s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:34.232 4076 3051114315096 MainThread ERROR nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Traceback (most recent call last): 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(*args, **kwargs) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] admin_password, network_info, block_device_info) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(self, *args, **kwds) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] should_plug_vifs=False) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'req_state': req_state}) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] time.sleep(sleep_time) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return f(*args, **kwargs) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return self._wait_for_job(job_path) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] error_desc=err_desc) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] During handling of the above exception, another exception occurred: 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Traceback (most recent call last): 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] accel_info=accel_info) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(*args, **kwargs) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] admin_password, network_info, block_device_info) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return function(self, *args, **kwds) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] should_plug_vifs=False) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'req_state': req_state}) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] time.sleep(sleep_time) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self.force_reraise() 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] raise self.value 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return f(*args, **kwargs) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] return self._wait_for_job(job_path) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] error_desc=err_desc) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. 2021-09-10 17:05:34.232 4076 3051114315096 MainThread TRACE nova.compute.manager [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG nova.compute.utils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Build of instance d9d8c32a-f534-4f8f-a1ba-5b0b5452598e was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000066' failed to start. Not enough memory in the system to start the virtual machine instance-00000066. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000066' failed to start. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Not enough memory in the system to start the virtual machine instance-00000066 with ram size 1024 megabytes. (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) 'instance-00000066' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D737F854-64D7-4FAC-BFCE-17EDF22C6FE7) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Acquired lock "refresh_cache-d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:34.232 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:34.295 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:34.326 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Successfully created port: 29258dff-8218-46ee-a2af-3a52e23f52ce _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:05:34.464 4076 3051114315096 MainThread DEBUG nova.network.neutron [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:34.468 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Releasing lock "refresh_cache-d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:34.468 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:34.468 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] [instance: d9d8c32a-f534-4f8f-a1ba-5b0b5452598e] Skipping network deallocation for instance since networking was not requested. _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2054 2021-09-10 17:05:34.774 4076 3051114315096 MainThread INFO nova.scheduler.client.report [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Deleted allocations for instance d9d8c32a-f534-4f8f-a1ba-5b0b5452598e 2021-09-10 17:05:34.802 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:34.802 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:05:34.850 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'B341ECFB-D506-445B-921B-A7B6B9F1580E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15412), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:34.877 4076 3051114315096 MainThread DEBUG oslo_concurrency.lockutils [req-e65109f7-0377-4238-90e5-effef713f29b ea6ef73c263247479859a70f94b17060 a93cbf4dbb764c2aba80a6d873876165 - default default] Lock "d9d8c32a-f534-4f8f-a1ba-5b0b5452598e" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 13.640s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:34.986 4076 3051093462184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:34.986 4076 3051093462184 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:34.986 4076 3051093461304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:34.986 4076 3051093461304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:34.986 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B341ECFB-D506-445B-921B-A7B6B9F1580E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112983), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:34.986 4076 3051114314568 MainThread DEBUG os_win._utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5188894271850586 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:35.016 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:35.016 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:05:35.045 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-aed87184-07ae-421e-bb73-955f16379f8b - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:35.045 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-aed87184-07ae-421e-bb73-955f16379f8b - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:35.093 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-aed87184-07ae-421e-bb73-955f16379f8b - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:35.118 4076 3051093462184 MainThread INFO nova.compute.manager [req-aed87184-07ae-421e-bb73-955f16379f8b - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:36.014 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'E539985E-AB41-44E6-908D-50D64D335B2E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14785), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:36.149 4076 3051093461128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:36.149 4076 3051093461128 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:36.161 4076 3051093461304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:36.161 4076 3051093461304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:36.161 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E539985E-AB41-44E6-908D-50D64D335B2E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118691), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:36.161 4076 3051114314568 MainThread DEBUG os_win._utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.694237470626831 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:36.189 4076 3051093461128 MainThread DEBUG nova.compute.manager [req-ad9cc053-15fb-4ab8-83e1-cc81e0304e18 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:36.189 4076 3051093461128 MainThread DEBUG nova.virt.hyperv.vmops [req-ad9cc053-15fb-4ab8-83e1-cc81e0304e18 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:36.237 4076 3051093461128 MainThread DEBUG nova.compute.manager [req-ad9cc053-15fb-4ab8-83e1-cc81e0304e18 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:36.264 4076 3051093461128 MainThread INFO nova.compute.manager [req-ad9cc053-15fb-4ab8-83e1-cc81e0304e18 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:36.422 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Successfully updated port: 29258dff-8218-46ee-a2af-3a52e23f52ce _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:05:36.462 4076 3051114315272 GreenThread-89 DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Acquired lock "refresh_cache-052c4e6a-de0f-4bd4-bfb2-812601b39335" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:36.462 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:36.524 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Received event network-changed-29258dff-8218-46ee-a2af-3a52e23f52ce external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:36.524 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Refreshing instance network info cache due to event network-changed-29258dff-8218-46ee-a2af-3a52e23f52ce. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:05:36.650 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:36.673 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:36.673 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:36.673 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:05:36.673 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:37.170 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'A26C846D-5AB0-4A36-ACA8-FF68AF15461B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14718), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:37.299 4076 3051114315096 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:37.299 4076 3051114315096 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:37.299 4076 3051093461128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:37.299 4076 3051093461128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:37.299 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A26C846D-5AB0-4A36-ACA8-FF68AF15461B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108379), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:37.299 4076 3051114314568 MainThread DEBUG os_win._utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.832272052764893 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:37.337 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-781adfae-ccb6-43ab-93bf-d13dbdcee66c - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:37.337 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-781adfae-ccb6-43ab-93bf-d13dbdcee66c - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:37.392 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-781adfae-ccb6-43ab-93bf-d13dbdcee66c - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:37.462 4076 3051114315096 MainThread INFO nova.compute.manager [req-781adfae-ccb6-43ab-93bf-d13dbdcee66c - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:37.462 4076 3051114315272 GreenThread-89 DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Updating instance_info_cache with network_info: [{"id": "29258dff-8218-46ee-a2af-3a52e23f52ce", "address": "fa:16:3e:29:d3:c0", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap29258dff-82", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:37.482 4076 3051114315272 GreenThread-89 DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Releasing lock "refresh_cache-052c4e6a-de0f-4bd4-bfb2-812601b39335" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:37.482 4076 3051114315272 GreenThread-89 DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Instance network_info: |[{"id": "29258dff-8218-46ee-a2af-3a52e23f52ce", "address": "fa:16:3e:29:d3:c0", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap29258dff-82", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:05:37.482 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Preparing to wait for external event network-vif-plugged-29258dff-8218-46ee-a2af-3a52e23f52ce prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:05:37.482 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:37.482 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:37.482 4076 3051114314920 MainThread DEBUG os_win.utils.compute.vmutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Creating VM instance-00000067 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:37.482 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-052c4e6a-de0f-4bd4-bfb2-812601b39335" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:37.482 4076 3051093462184 MainThread DEBUG nova.network.neutron [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Refreshing network info cache for port 29258dff-8218-46ee-a2af-3a52e23f52ce _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:05:38.373 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '11CB4188-341B-4D4D-83CD-D951520F06C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16189), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:38.391 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:05:38.483 4076 3051114315096 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:38.483 4076 3051114315096 MainThread INFO nova.compute.manager [-] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] VM Stopped (Lifecycle Event) 2021-09-10 17:05:38.483 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:38.483 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:38.504 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000063\' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F12399510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000063\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)instance-0000006340D1640F-25C8-427F-A043-529E1F123995%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '11CB4188-341B-4D4D-83CD-D951520F06C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114513), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\nNot enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)\n\n'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995)", 'ErrorSummaryDescription': "'instance-00000063' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000063.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:38.504 4076 3051114314568 MainThread ERROR nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-00000063 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. 2021-09-10 17:05:38.504 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Got request to destroy instance 2021-09-10 17:05:38.536 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-7c0d0317-03bf-4ab9-bbad-b6ec13a74686 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:38.536 4076 3051114315096 MainThread DEBUG nova.virt.hyperv.vmops [req-7c0d0317-03bf-4ab9-bbad-b6ec13a74686 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:38.598 4076 3051114315096 MainThread DEBUG nova.compute.manager [req-7c0d0317-03bf-4ab9-bbad-b6ec13a74686 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:38.635 4076 3051114315096 MainThread INFO nova.compute.manager [req-7c0d0317-03bf-4ab9-bbad-b6ec13a74686 - - - - -] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:38.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:38.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:38.994 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:39.092 4076 3051093462184 MainThread DEBUG nova.network.neutron [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Updated VIF entry in instance network info cache for port 29258dff-8218-46ee-a2af-3a52e23f52ce. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:05:39.092 4076 3051093462184 MainThread DEBUG nova.network.neutron [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Updating instance_info_cache with network_info: [{"id": "29258dff-8218-46ee-a2af-3a52e23f52ce", "address": "fa:16:3e:29:d3:c0", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap29258dff-82", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:39.125 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-6e0987fb-3d57-42e4-8a6a-cf87d0ab23a6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-052c4e6a-de0f-4bd4-bfb2-812601b39335" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:40.327 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:40.327 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:40.327 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:40.357 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-00000063 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:40.357 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-00000063 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:40.389 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'D4098D7C-74D5-4352-BC4E-839785C86F51', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14664), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:40.493 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': 'D4098D7C-74D5-4352-BC4E-839785C86F51', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32540), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:40.493 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:40.493 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-00000063 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:40.513 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:40.514 4076 3051114314568 MainThread ERROR nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Traceback (most recent call last): 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(*args, **kwargs) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] admin_password, network_info, block_device_info) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(self, *args, **kwds) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] should_plug_vifs=False) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'req_state': req_state}) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] time.sleep(sleep_time) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return f(*args, **kwargs) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return self._wait_for_job(job_path) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] error_desc=err_desc) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During handling of the above exception, another exception occurred: 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Traceback (most recent call last): 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] yield resources 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] accel_info=accel_info) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(*args, **kwargs) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] admin_password, network_info, block_device_info) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(self, *args, **kwds) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] should_plug_vifs=False) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'req_state': req_state}) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] time.sleep(sleep_time) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return f(*args, **kwargs) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return self._wait_for_job(job_path) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] error_desc=err_desc) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.514 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.514 4076 3051114314568 MainThread INFO nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Terminating instance 2021-09-10 17:05:40.514 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:40.514 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Got request to destroy instance 2021-09-10 17:05:40.514 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:40.514 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:40.514 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000063" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:40.514 4076 3051114314568 MainThread INFO nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:40.540 4076 3051114314568 MainThread DEBUG nova.compute.claims [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:40.540 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:40.685 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:40.685 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:40.810 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:40.896 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:40.914 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:40.914 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:40.914 4076 3051114314568 MainThread ERROR nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Traceback (most recent call last): 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(*args, **kwargs) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] admin_password, network_info, block_device_info) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(self, *args, **kwds) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] should_plug_vifs=False) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'req_state': req_state}) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] time.sleep(sleep_time) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return f(*args, **kwargs) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return self._wait_for_job(job_path) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] error_desc=err_desc) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] During handling of the above exception, another exception occurred: 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Traceback (most recent call last): 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] accel_info=accel_info) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(*args, **kwargs) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] admin_password, network_info, block_device_info) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return function(self, *args, **kwds) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] should_plug_vifs=False) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'req_state': req_state}) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] time.sleep(sleep_time) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self.force_reraise() 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] raise self.value 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return f(*args, **kwargs) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] return self._wait_for_job(job_path) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] error_desc=err_desc) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. 2021-09-10 17:05:40.914 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.compute.utils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Build of instance e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000063' failed to start. Not enough memory in the system to start the virtual machine instance-00000063. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000063' failed to start. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Not enough memory in the system to start the virtual machine instance-00000063 with ram size 1024 megabytes. (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) 'instance-00000063' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 40D1640F-25C8-427F-A043-529E1F123995) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:05:40.929 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:05:41.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:41.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:41.806 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:41.806 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:41.806 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:05:41.806 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:41.821 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:42.040 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2188MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:05:42.040 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:42.234 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:42.293 4076 3051114314568 MainThread INFO nova.compute.manager [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca] Took 1.36 seconds to deallocate network for instance. 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.343 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 17:05:42.343 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 052c4e6a-de0f-4bd4-bfb2-812601b39335 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:05:42.343 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:05:42.343 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:05:42.377 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:42.405 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:42.514 4076 3051114314568 MainThread INFO nova.scheduler.client.report [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca 2021-09-10 17:05:42.560 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-4d67209d-a5d7-49d6-a37b-004d9f98c900 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "e7ea83ca-2fbc-4678-85d8-8dd8f348c7ca" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 23.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:42.610 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:42.665 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:42.665 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:05:42.665 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:43.399 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-9334a1cb-3962-460d-8c94-b9986dc2dd51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Received event network-vif-plugged-29258dff-8218-46ee-a2af-3a52e23f52ce external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:43.399 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-9334a1cb-3962-460d-8c94-b9986dc2dd51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:43.399 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-9334a1cb-3962-460d-8c94-b9986dc2dd51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:43.405 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-9334a1cb-3962-460d-8c94-b9986dc2dd51 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Processing event network-vif-plugged-29258dff-8218-46ee-a2af-3a52e23f52ce _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:05:43.482 4076 3051114314920 MainThread INFO nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Using config drive for instance 2021-09-10 17:05:43.696 4076 3051114314920 MainThread INFO nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Creating config drive at C:\openstack\instances\instance-00000067\configdrive.iso 2021-09-10 17:05:43.750 4076 3051114314920 MainThread DEBUG oslo_concurrency.processutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000067\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpcxy_n7sk execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:05:44.076 4076 3051114314920 MainThread DEBUG oslo_concurrency.processutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000067\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpcxy_n7sk" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:05:44.455 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Setting boot order for instance: instance-00000067: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:05:44.500 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:05:44.544 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': 'BA7994EC-F9B1-4687-9C77-91C9C63D268B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14265), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:44.657 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:44.657 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:44.681 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:44.681 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:44.681 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:44.681 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:05:44.681 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BA7994EC-F9B1-4687-9C77-91C9C63D268B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112983), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:44.698 4076 3051114314920 MainThread DEBUG os_win._utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19807028770446777 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:44.706 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-200f054a-7cd1-40f2-b416-948c0424558e - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:44.706 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-200f054a-7cd1-40f2-b416-948c0424558e - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:44.743 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-200f054a-7cd1-40f2-b416-948c0424558e - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:44.787 4076 3051113162712 MainThread INFO nova.compute.manager [req-200f054a-7cd1-40f2-b416-948c0424558e - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:45.726 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '00FEA379-1630-432E-90B1-06326CD14B93', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15617), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:45.842 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:45.842 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:45.842 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:45.842 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:45.870 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '00FEA379-1630-432E-90B1-06326CD14B93', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110402), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:45.870 4076 3051114314920 MainThread DEBUG os_win._utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3694360256195068 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:45.888 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-8994f40a-df30-4ea0-b454-ee69bdc146f5 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:45.888 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-8994f40a-df30-4ea0-b454-ee69bdc146f5 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:45.926 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-8994f40a-df30-4ea0-b454-ee69bdc146f5 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:45.982 4076 3051113162712 MainThread INFO nova.compute.manager [req-8994f40a-df30-4ea0-b454-ee69bdc146f5 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:46.790 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:05:46.893 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '5D3C8615-1B63-4D5D-AB3F-4A07BE49AA07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14872), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:47.014 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:47.014 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:47.014 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:47.014 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:47.045 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5D3C8615-1B63-4D5D-AB3F-4A07BE49AA07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113501), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:47.045 4076 3051114314920 MainThread DEBUG os_win._utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.544240713119507 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:47.061 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-37ee9550-cb51-4afd-a75c-f447c9fdc2d0 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:47.061 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-37ee9550-cb51-4afd-a75c-f447c9fdc2d0 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:47.107 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-37ee9550-cb51-4afd-a75c-f447c9fdc2d0 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:47.163 4076 3051113162712 MainThread INFO nova.compute.manager [req-37ee9550-cb51-4afd-a75c-f447c9fdc2d0 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:48.076 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '79FFE16C-D577-4F45-86E7-7BC6C4445F09', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14593), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:48.187 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:48.187 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:48.187 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:48.187 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:48.219 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '79FFE16C-D577-4F45-86E7-7BC6C4445F09', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109743), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:48.219 4076 3051114314920 MainThread DEBUG os_win._utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7184255123138428 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:48.248 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-051c7101-6f22-4669-814e-c8a510b73e1c - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:48.248 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-051c7101-6f22-4669-814e-c8a510b73e1c - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:48.290 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-051c7101-6f22-4669-814e-c8a510b73e1c - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:48.326 4076 3051113162712 MainThread INFO nova.compute.manager [req-051c7101-6f22-4669-814e-c8a510b73e1c - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:49.257 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '5CCF3D66-D34F-41C6-97B1-99D93CC114A8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14509), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:49.383 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:49.392 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:49.392 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:49.392 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:49.392 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5CCF3D66-D34F-41C6-97B1-99D93CC114A8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110843), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:49.410 4076 3051114314920 MainThread DEBUG os_win._utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.891966104507446 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:05:49.421 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-1a875baa-c9dd-46c6-98a2-f5ee7ac1b844 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:49.421 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-1a875baa-c9dd-46c6-98a2-f5ee7ac1b844 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:49.466 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-1a875baa-c9dd-46c6-98a2-f5ee7ac1b844 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:49.508 4076 3051113162712 MainThread INFO nova.compute.manager [req-1a875baa-c9dd-46c6-98a2-f5ee7ac1b844 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:50.439 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': 'E44E985A-8E47-41CF-9E59-8E3E422421AF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15743), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:50.560 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:05:50.584 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] VM Stopped (Lifecycle Event) 2021-09-10 17:05:50.584 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:50.584 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:50.584 4076 3051114314920 MainThread ERROR os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000067\' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F1910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000067\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)instance-000000670451D43E-7EF8-4BA7-8A96-4DDF290C0F19%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E44E985A-8E47-41CF-9E59-8E3E422421AF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111220), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 5, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 5, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\nNot enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)\n\n'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19)", 'ErrorSummaryDescription': "'instance-00000067' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000067.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:05:50.584 4076 3051114314920 MainThread ERROR nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Failed to change vm state of instance-00000067 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. 2021-09-10 17:05:50.584 4076 3051114314920 MainThread INFO nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Got request to destroy instance 2021-09-10 17:05:50.618 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-f57d597d-5854-4a66-8b42-c50604a20a76 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:05:50.618 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-f57d597d-5854-4a66-8b42-c50604a20a76 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:05:50.664 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-f57d597d-5854-4a66-8b42-c50604a20a76 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:05:50.698 4076 3051113162712 MainThread INFO nova.compute.manager [req-f57d597d-5854-4a66-8b42-c50604a20a76 - - - - -] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:05:51.479 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:51.529 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:51.676 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:51.676 4076 3051113161128 MainThread DEBUG nova.virt.hardware [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:51.676 4076 3051113161128 MainThread INFO nova.compute.claims [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Claim successful on node n-h1-808474-1 2021-09-10 17:05:51.978 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:51.988 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:52.196 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:52.216 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:52.216 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.216 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:52.361 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:05:52.361 4076 3051113162712 GreenThread-92 DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:05:52.361 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:05:52.404 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:52.466 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:05:52.466 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.466 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.485 4076 3051114314920 MainThread DEBUG os_win.utils.compute.vmutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Successfully changed vm state of instance-00000067 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:05:52.485 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Successfully changed state of VM instance-00000067 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:05:52.540 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': 'E12A97EC-8DD7-4081-96D3-3E0302E2BF84', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15911), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:05:52.661 4076 3051114314920 MainThread DEBUG os_win.utils.jobutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job succeeded: {'InstanceID': 'E12A97EC-8DD7-4081-96D3-3E0302E2BF84', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29360), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:05:52.661 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.661 4076 3051114314920 MainThread DEBUG os_win.utils.pathutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Removing directory: C:\openstack\instances\instance-00000067 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:05:52.661 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.661 4076 3051114314920 MainThread ERROR nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Traceback (most recent call last): 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(*args, **kwargs) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] admin_password, network_info, block_device_info) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(self, *args, **kwds) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] should_plug_vifs=False) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'req_state': req_state}) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] time.sleep(sleep_time) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return f(*args, **kwargs) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return self._wait_for_job(job_path) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] error_desc=err_desc) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During handling of the above exception, another exception occurred: 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Traceback (most recent call last): 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] yield resources 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] accel_info=accel_info) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(*args, **kwargs) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] admin_password, network_info, block_device_info) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(self, *args, **kwds) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] should_plug_vifs=False) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'req_state': req_state}) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] time.sleep(sleep_time) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return f(*args, **kwargs) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return self._wait_for_job(job_path) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] error_desc=err_desc) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:52.661 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:52.661 4076 3051114314920 MainThread INFO nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Terminating instance 2021-09-10 17:05:52.675 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:05:52.675 4076 3051114314920 MainThread INFO nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Got request to destroy instance 2021-09-10 17:05:52.675 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:05:52.693 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:05:52.693 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.693 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-00000067" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.693 4076 3051114314920 MainThread INFO nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:05:52.693 4076 3051114314920 MainThread DEBUG nova.compute.claims [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:05:52.693 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.693 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Spawning new instance 2021-09-10 17:05:52.693 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.693 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.693 4076 3051113161128 MainThread DEBUG nova.block_device [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:05:52.716 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:52.716 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:52.721 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating directory: C:\openstack\instances\instance-00000069 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:05:52.721 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000069\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:05:52.749 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Resizing VHD C:\openstack\instances\instance-00000069\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:05:52.889 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.hostops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:52.908 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.hostops [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:52.946 4076 3051113162712 GreenThread-92 DEBUG nova.policy [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '9f286737f0e6452bbe04f42184b6d12d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df662acac0f043bb8be9c87490878d93', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:05:53.125 4076 3051114314920 MainThread DEBUG nova.compute.provider_tree [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:53.125 4076 3051114314920 MainThread DEBUG nova.scheduler.client.report [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:53.125 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:53.125 4076 3051114314920 MainThread ERROR nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Traceback (most recent call last): 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(*args, **kwargs) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] admin_password, network_info, block_device_info) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(self, *args, **kwds) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] should_plug_vifs=False) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'req_state': req_state}) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] time.sleep(sleep_time) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return f(*args, **kwargs) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return self._wait_for_job(job_path) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] error_desc=err_desc) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] During handling of the above exception, another exception occurred: 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Traceback (most recent call last): 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] accel_info=accel_info) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(*args, **kwargs) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] admin_password, network_info, block_device_info) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return function(self, *args, **kwds) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] should_plug_vifs=False) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'req_state': req_state}) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] time.sleep(sleep_time) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self.force_reraise() 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] raise self.value 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return f(*args, **kwargs) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] return self._wait_for_job(job_path) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] error_desc=err_desc) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. 2021-09-10 17:05:53.125 4076 3051114314920 MainThread TRACE nova.compute.manager [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.compute.utils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Build of instance 052c4e6a-de0f-4bd4-bfb2-812601b39335 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000067' failed to start. Not enough memory in the system to start the virtual machine instance-00000067. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000067' failed to start. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Not enough memory in the system to start the virtual machine instance-00000067 with ram size 1024 megabytes. (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) 'instance-00000067' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 0451D43E-7EF8-4BA7-8A96-4DDF290C0F19) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:05:53.149 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:05:55.337 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Successfully created port: 9e661436-37b1-4e90-a784-f6da864ff530 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:05:55.701 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:55.727 4076 3051114314920 MainThread INFO nova.compute.manager [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 052c4e6a-de0f-4bd4-bfb2-812601b39335] Took 2.58 seconds to deallocate network for instance. 2021-09-10 17:05:56.086 4076 3051114314920 MainThread INFO nova.scheduler.client.report [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Deleted allocations for instance 052c4e6a-de0f-4bd4-bfb2-812601b39335 2021-09-10 17:05:56.126 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-00650ea2-eede-4b29-8f6b-afdf11773bfe 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "052c4e6a-de0f-4bd4-bfb2-812601b39335" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:58.038 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Successfully updated port: 9e661436-37b1-4e90-a784-f6da864ff530 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:05:58.071 4076 3051113162712 GreenThread-92 DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Acquired lock "refresh_cache-4e15be49-462b-4156-9978-5513db0f17ba" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:58.071 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:05:58.329 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Received event network-changed-9e661436-37b1-4e90-a784-f6da864ff530 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:05:58.329 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Refreshing instance network info cache due to event network-changed-9e661436-37b1-4e90-a784-f6da864ff530. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:05:58.360 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:05:59.009 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:59.015 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:05:59.232 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:59.232 4076 3051114314568 MainThread DEBUG nova.virt.hardware [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:05:59.232 4076 3051114314568 MainThread INFO nova.compute.claims [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Claim successful on node n-h1-808474-1 2021-09-10 17:05:59.455 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:05:59.482 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:05:59.675 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:05:59.721 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:05:59.721 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:59.721 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:05:59.853 4076 3051114314568 MainThread DEBUG nova.compute.utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:05:59.860 4076 3051114315272 GreenThread-93 DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:05:59.860 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:05:59.888 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:05:59.919 4076 3051113162712 GreenThread-92 DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Updating instance_info_cache with network_info: [{"id": "9e661436-37b1-4e90-a784-f6da864ff530", "address": "fa:16:3e:c1:8a:59", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9e661436-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:05:59.953 4076 3051113162712 GreenThread-92 DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Releasing lock "refresh_cache-4e15be49-462b-4156-9978-5513db0f17ba" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:05:59.953 4076 3051113162712 GreenThread-92 DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Instance network_info: |[{"id": "9e661436-37b1-4e90-a784-f6da864ff530", "address": "fa:16:3e:c1:8a:59", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9e661436-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:05:59.953 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Preparing to wait for external event network-vif-plugged-9e661436-37b1-4e90-a784-f6da864ff530 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:05:59.953 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:05:59.953 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:05:59.953 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Creating VM instance-00000069 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:05:59.953 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4e15be49-462b-4156-9978-5513db0f17ba" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:05:59.953 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Refreshing network info cache for port 9e661436-37b1-4e90-a784-f6da864ff530 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:06:00.185 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:06:00.185 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Spawning new instance 2021-09-10 17:06:00.208 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:00.208 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:00.208 4076 3051114314568 MainThread DEBUG nova.block_device [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:06:00.208 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:00.208 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:00.217 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating directory: C:\openstack\instances\instance-0000006a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:06:00.217 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:06:00.233 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Resizing VHD C:\openstack\instances\instance-0000006a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:06:00.654 4076 3051114315272 GreenThread-93 DEBUG nova.policy [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6da982fe4daa44b998cd4ae6900212fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0928f57ccc9245e28c79dd9cce7e395b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:06:00.841 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:06:01.252 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Updated VIF entry in instance network info cache for port 9e661436-37b1-4e90-a784-f6da864ff530. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:06:01.252 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Updating instance_info_cache with network_info: [{"id": "9e661436-37b1-4e90-a784-f6da864ff530", "address": "fa:16:3e:c1:8a:59", "network": {"id": "5b608de7-ebcb-494c-bcfe-060ce4426ed6", "bridge": null, "label": "tempest-ServersTestJSON-497725561-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "df662acac0f043bb8be9c87490878d93", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9e661436-37", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:01.263 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-72acc171-e4f5-4ded-8808-f33c7fafc15e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4e15be49-462b-4156-9978-5513db0f17ba" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:03.419 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Successfully created port: a41c6c93-c50a-45fb-bb0c-808b88dbfeca _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:06:07.145 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Successfully updated port: a41c6c93-c50a-45fb-bb0c-808b88dbfeca _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:06:07.178 4076 3051114315272 GreenThread-93 DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Acquired lock "refresh_cache-fc38b331-efcd-4c62-aecf-3859bffff0bb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:07.178 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:06:07.435 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Received event network-changed-a41c6c93-c50a-45fb-bb0c-808b88dbfeca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:07.435 4076 3051114312280 MainThread DEBUG nova.compute.manager [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Refreshing instance network info cache due to event network-changed-a41c6c93-c50a-45fb-bb0c-808b88dbfeca. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:06:07.623 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:06:08.452 4076 3051114315272 GreenThread-93 DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Updating instance_info_cache with network_info: [{"id": "a41c6c93-c50a-45fb-bb0c-808b88dbfeca", "address": "fa:16:3e:8b:42:ab", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa41c6c93-c5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:08.490 4076 3051114315272 GreenThread-93 DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Releasing lock "refresh_cache-fc38b331-efcd-4c62-aecf-3859bffff0bb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:08.490 4076 3051114315272 GreenThread-93 DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Instance network_info: |[{"id": "a41c6c93-c50a-45fb-bb0c-808b88dbfeca", "address": "fa:16:3e:8b:42:ab", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa41c6c93-c5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:06:08.490 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Preparing to wait for external event network-vif-plugged-a41c6c93-c50a-45fb-bb0c-808b88dbfeca prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:06:08.490 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:08.490 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:08.490 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating VM instance-0000006a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:06:08.490 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-fc38b331-efcd-4c62-aecf-3859bffff0bb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:08.490 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Refreshing network info cache for port a41c6c93-c50a-45fb-bb0c-808b88dbfeca _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:06:08.518 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-f916a9cb-2cde-4884-af2d-85e431036c64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Received event network-vif-plugged-9e661436-37b1-4e90-a784-f6da864ff530 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:08.518 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [req-f916a9cb-2cde-4884-af2d-85e431036c64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:08.518 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [req-f916a9cb-2cde-4884-af2d-85e431036c64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:08.518 4076 3051114315272 MainThread DEBUG nova.compute.manager [req-f916a9cb-2cde-4884-af2d-85e431036c64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Processing event network-vif-plugged-9e661436-37b1-4e90-a784-f6da864ff530 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:06:08.546 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Using config drive for instance 2021-09-10 17:06:09.345 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Creating config drive at C:\openstack\instances\instance-00000069\configdrive.iso 2021-09-10 17:06:09.360 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000069\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp38e_s51b execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:06:09.565 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:06:09.700 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000069\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp38e_s51b" returned: 0 in 0.343s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:06:09.969 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Updated VIF entry in instance network info cache for port a41c6c93-c50a-45fb-bb0c-808b88dbfeca. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:06:09.969 4076 3051114312280 MainThread DEBUG nova.network.neutron [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Updating instance_info_cache with network_info: [{"id": "a41c6c93-c50a-45fb-bb0c-808b88dbfeca", "address": "fa:16:3e:8b:42:ab", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa41c6c93-c5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:09.982 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [req-d903d721-647f-4375-bb0c-d2988e61104e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-fc38b331-efcd-4c62-aecf-3859bffff0bb" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:10.095 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Setting boot order for instance: instance-00000069: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:06:10.139 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:06:10.162 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '1429F62F-3C39-42D0-9C22-0A6546E4A933', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14602), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:10.310 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:10.310 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:10.310 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:10.321 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:10.321 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1429F62F-3C39-42D0-9C22-0A6546E4A933', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111336), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:10.321 4076 3051113161128 MainThread DEBUG os_win._utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18181252479553223 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:10.336 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-5a08a7df-fa44-4703-9bdf-6ee9f2841127 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:10.336 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-5a08a7df-fa44-4703-9bdf-6ee9f2841127 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:10.389 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-5a08a7df-fa44-4703-9bdf-6ee9f2841127 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:10.451 4076 3051113162712 MainThread INFO nova.compute.manager [req-5a08a7df-fa44-4703-9bdf-6ee9f2841127 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:11.373 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '25B8604B-732B-4B78-8891-33473E47D801', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15606), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:11.483 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:11.508 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:11.508 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:11.508 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:11.508 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '25B8604B-732B-4B78-8891-33473E47D801', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111562), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:11.508 4076 3051113161128 MainThread DEBUG os_win._utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3689813613891602 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:11.527 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-3d9f0240-ef67-4b33-bd8c-da24563a6064 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:11.527 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-3d9f0240-ef67-4b33-bd8c-da24563a6064 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:11.566 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-3d9f0240-ef67-4b33-bd8c-da24563a6064 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:11.615 4076 3051113162712 MainThread INFO nova.compute.manager [req-3d9f0240-ef67-4b33-bd8c-da24563a6064 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:12.555 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '931538BE-B87A-4FDC-B97D-9D36EDE62953', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15081), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:12.686 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:12.686 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:12.686 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:12.686 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:12.686 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '931538BE-B87A-4FDC-B97D-9D36EDE62953', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112229), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:12.711 4076 3051113161128 MainThread DEBUG os_win._utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.547142505645752 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:12.720 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-78ea7c0e-93bb-49e5-b7d1-9c25eb7012cf - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:12.720 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-78ea7c0e-93bb-49e5-b7d1-9c25eb7012cf - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:12.773 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-78ea7c0e-93bb-49e5-b7d1-9c25eb7012cf - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:12.805 4076 3051113162712 MainThread INFO nova.compute.manager [req-78ea7c0e-93bb-49e5-b7d1-9c25eb7012cf - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:13.758 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '82A02505-90D1-4630-B56F-DAA16AA69563', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15357), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:13.888 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:13.888 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:13.888 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:13.888 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:13.910 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-131349fd-8ad6-4816-b0c6-6aaff23f5c96 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:13.920 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-131349fd-8ad6-4816-b0c6-6aaff23f5c96 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:13.920 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '82A02505-90D1-4630-B56F-DAA16AA69563', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111151), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:13.920 4076 3051113161128 MainThread DEBUG os_win._utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.78106951713562 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:13.966 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-131349fd-8ad6-4816-b0c6-6aaff23f5c96 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:14.013 4076 3051113162712 MainThread INFO nova.compute.manager [req-131349fd-8ad6-4816-b0c6-6aaff23f5c96 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:14.945 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '1FA4B175-439C-448A-8290-71B60B5002B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14934), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:15.079 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:15.079 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:15.079 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:15.079 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:15.092 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1FA4B175-439C-448A-8290-71B60B5002B8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111095), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:15.092 4076 3051113161128 MainThread DEBUG os_win._utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9525227546691895 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:15.117 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-b8eb3f41-7c34-4563-8d1f-2c8d73894ee2 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:15.117 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-b8eb3f41-7c34-4563-8d1f-2c8d73894ee2 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:15.144 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-b8eb3f41-7c34-4563-8d1f-2c8d73894ee2 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:15.195 4076 3051113162712 MainThread INFO nova.compute.manager [req-b8eb3f41-7c34-4563-8d1f-2c8d73894ee2 - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:16.130 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': '822B7017-B63E-4A1D-B77F-AE1A7A867047', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15194), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:16.235 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-bec16f42-6f25-4e74-9545-589ebb4eab70 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Received event network-vif-plugged-a41c6c93-c50a-45fb-bb0c-808b88dbfeca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:16.235 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-bec16f42-6f25-4e74-9545-589ebb4eab70 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:16.235 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [req-bec16f42-6f25-4e74-9545-589ebb4eab70 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:16.235 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-bec16f42-6f25-4e74-9545-589ebb4eab70 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Processing event network-vif-plugged-a41c6c93-c50a-45fb-bb0c-808b88dbfeca _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:06:16.248 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:16.265 4076 3051114312280 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:16.265 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:16.265 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] VM Stopped (Lifecycle Event) 2021-09-10 17:06:16.265 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000069\' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62012030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F62010243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000069\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)instance-000000692E5CE86D-8AA4-4C92-A493-D25E2361F620%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '822B7017-B63E-4A1D-B77F-AE1A7A867047', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109795), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\nNot enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)\n\n'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620)", 'ErrorSummaryDescription': "'instance-00000069' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000069.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:16.265 4076 3051113161128 MainThread ERROR nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Failed to change vm state of instance-00000069 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. 2021-09-10 17:06:16.265 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Got request to destroy instance 2021-09-10 17:06:16.265 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Using config drive for instance 2021-09-10 17:06:16.348 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-2aeda614-e64c-44b5-83ad-e77a2f40128e - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:16.348 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-2aeda614-e64c-44b5-83ad-e77a2f40128e - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:16.389 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-2aeda614-e64c-44b5-83ad-e77a2f40128e - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:16.442 4076 3051113162712 MainThread INFO nova.compute.manager [req-2aeda614-e64c-44b5-83ad-e77a2f40128e - - - - -] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:17.133 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Creating config drive at C:\openstack\instances\instance-0000006a\configdrive.iso 2021-09-10 17:06:17.169 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpr9guakn9 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:06:17.470 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpr9guakn9" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:06:17.891 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Setting boot order for instance: instance-0000006a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:06:17.941 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:06:17.972 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'CCE1A8C2-8A02-408D-89EC-8BECA7970BE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14836), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:18.110 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:18.110 4076 3051114315272 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:18.110 4076 3051113162712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:18.110 4076 3051113162712 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:18.110 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CCE1A8C2-8A02-408D-89EC-8BECA7970BE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:18.110 4076 3051114314568 MainThread DEBUG os_win._utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16949701309204102 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:18.138 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-b219a231-7cb5-4a1c-83da-8bec1e9f6be6 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:18.138 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [req-b219a231-7cb5-4a1c-83da-8bec1e9f6be6 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:18.180 4076 3051113162712 MainThread DEBUG nova.compute.manager [req-b219a231-7cb5-4a1c-83da-8bec1e9f6be6 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:18.211 4076 3051113162712 MainThread INFO nova.compute.manager [req-b219a231-7cb5-4a1c-83da-8bec1e9f6be6 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:18.310 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:06:18.310 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:18.310 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:18.320 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed vm state of instance-00000069 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:06:18.320 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Successfully changed state of VM instance-00000069 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:06:18.375 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Waiting for WMI job: {'InstanceID': 'B1637B58-A66D-48F1-9E07-F862C8358C30', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15591), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:18.484 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] WMI job succeeded: {'InstanceID': 'B1637B58-A66D-48F1-9E07-F862C8358C30', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29127), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:06:18.484 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:18.484 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Removing directory: C:\openstack\instances\instance-00000069 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:06:18.501 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:18.501 4076 3051113161128 MainThread ERROR nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Traceback (most recent call last): 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(*args, **kwargs) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] admin_password, network_info, block_device_info) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(self, *args, **kwds) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] should_plug_vifs=False) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'req_state': req_state}) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] time.sleep(sleep_time) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return f(*args, **kwargs) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return self._wait_for_job(job_path) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] error_desc=err_desc) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During handling of the above exception, another exception occurred: 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Traceback (most recent call last): 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] yield resources 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] accel_info=accel_info) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(*args, **kwargs) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] admin_password, network_info, block_device_info) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(self, *args, **kwds) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] should_plug_vifs=False) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'req_state': req_state}) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] time.sleep(sleep_time) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return f(*args, **kwargs) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return self._wait_for_job(job_path) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] error_desc=err_desc) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:18.501 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:18.501 4076 3051113161128 MainThread INFO nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Terminating instance 2021-09-10 17:06:18.501 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:06:18.501 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Got request to destroy instance 2021-09-10 17:06:18.523 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:06:18.523 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:18.523 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "instance-00000069" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:18.523 4076 3051113161128 MainThread INFO nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:06:18.523 4076 3051113161128 MainThread DEBUG nova.compute.claims [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:06:18.530 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:18.731 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:18.752 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:18.941 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:19.000 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:19.000 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:19.017 4076 3051113161128 MainThread ERROR nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Traceback (most recent call last): 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(*args, **kwargs) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] admin_password, network_info, block_device_info) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(self, *args, **kwds) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] should_plug_vifs=False) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'req_state': req_state}) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] time.sleep(sleep_time) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return f(*args, **kwargs) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return self._wait_for_job(job_path) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] error_desc=err_desc) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] During handling of the above exception, another exception occurred: 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Traceback (most recent call last): 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] accel_info=accel_info) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(*args, **kwargs) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] admin_password, network_info, block_device_info) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return function(self, *args, **kwds) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] should_plug_vifs=False) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'req_state': req_state}) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] time.sleep(sleep_time) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self.force_reraise() 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] raise self.value 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return f(*args, **kwargs) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] return self._wait_for_job(job_path) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] error_desc=err_desc) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. 2021-09-10 17:06:19.017 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 4e15be49-462b-4156-9978-5513db0f17ba] 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Build of instance 4e15be49-462b-4156-9978-5513db0f17ba was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000069' failed to start. Not enough memory in the system to start the virtual machine instance-00000069. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000069' failed to start. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Not enough memory in the system to start the virtual machine instance-00000069 with ram size 1024 megabytes. (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) 'instance-00000069' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2E5CE86D-8AA4-4C92-A493-D25E2361F620) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:06:19.017 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:06:19.107 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:19.131 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '75ADDAF6-DE2A-4A3F-B02C-CA1A4E7A3FDC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15633), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:19.144 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:06:19.268 4076 3051093462536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:19.268 4076 3051093462536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:19.268 4076 3051093462008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:19.268 4076 3051093462008 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:19.280 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '75ADDAF6-DE2A-4A3F-B02C-CA1A4E7A3FDC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110851), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:19.280 4076 3051114314568 MainThread DEBUG os_win._utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.338587999343872 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:19.326 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-948a402f-e0a6-4278-bed5-dbfa0c2487bc - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:19.326 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-948a402f-e0a6-4278-bed5-dbfa0c2487bc - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:19.357 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:19.357 4076 3051093462360 MainThread DEBUG nova.virt.hardware [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:06:19.357 4076 3051093462360 MainThread INFO nova.compute.claims [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Claim successful on node n-h1-808474-1 2021-09-10 17:06:19.385 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-948a402f-e0a6-4278-bed5-dbfa0c2487bc - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:19.419 4076 3051093462008 MainThread INFO nova.compute.manager [req-948a402f-e0a6-4278-bed5-dbfa0c2487bc - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:19.602 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.hostops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:19.609 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.hostops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:19.809 4076 3051093462360 MainThread DEBUG nova.compute.provider_tree [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:19.828 4076 3051093462360 MainThread DEBUG nova.scheduler.client.report [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:19.828 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:19.828 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:06:19.961 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:06:19.986 4076 3051093462008 GreenThread-94 DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:06:19.986 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:06:20.295 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:06:20.295 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '02634575-5669-4D59-84C1-C72AD54C3FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14979), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:20.295 4076 3051093462360 MainThread INFO nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Spawning new instance 2021-09-10 17:06:20.316 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:20.316 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:20.316 4076 3051093462360 MainThread DEBUG nova.block_device [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:06:20.316 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:20.331 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:20.331 4076 3051093462008 GreenThread-94 DEBUG nova.policy [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:06:20.342 4076 3051093462360 MainThread DEBUG os_win.utils.pathutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-00000068 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:06:20.342 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000068\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:06:20.360 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Resizing VHD C:\openstack\instances\instance-00000068\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:06:20.435 4076 3051093462888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:20.435 4076 3051093462888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:20.435 4076 3051093462712 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:20.435 4076 3051093462712 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:20.456 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '02634575-5669-4D59-84C1-C72AD54C3FCD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120005), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:20.456 4076 3051114314568 MainThread DEBUG os_win._utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.515310525894165 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:20.483 4076 3051093462712 MainThread DEBUG nova.compute.manager [req-d09251b6-73df-4e04-a3a8-bc75e7636c90 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:20.483 4076 3051093462712 MainThread DEBUG nova.virt.hyperv.vmops [req-d09251b6-73df-4e04-a3a8-bc75e7636c90 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:20.528 4076 3051093462712 MainThread DEBUG nova.compute.manager [req-d09251b6-73df-4e04-a3a8-bc75e7636c90 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:20.562 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:20.562 4076 3051093462712 MainThread INFO nova.compute.manager [req-d09251b6-73df-4e04-a3a8-bc75e7636c90 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:20.609 4076 3051113161128 MainThread INFO nova.compute.manager [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] [instance: 4e15be49-462b-4156-9978-5513db0f17ba] Took 1.59 seconds to deallocate network for instance. 2021-09-10 17:06:20.950 4076 3051113161128 MainThread INFO nova.scheduler.client.report [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Deleted allocations for instance 4e15be49-462b-4156-9978-5513db0f17ba 2021-09-10 17:06:21.017 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-329f875d-6f00-4cfc-a3ca-59760b1061da 9f286737f0e6452bbe04f42184b6d12d df662acac0f043bb8be9c87490878d93 - default default] Lock "4e15be49-462b-4156-9978-5513db0f17ba" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:21.482 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '572296D7-4819-45F2-88FB-4F1F500F9962', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21656), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:21.612 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:21.612 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:21.612 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:21.612 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:21.631 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '572296D7-4819-45F2-88FB-4F1F500F9962', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114364), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:21.631 4076 3051114314568 MainThread DEBUG os_win._utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6897192001342773 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:21.655 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-fc127039-e359-4212-83f6-32b9f5af116d - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:21.655 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-fc127039-e359-4212-83f6-32b9f5af116d - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:21.703 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-fc127039-e359-4212-83f6-32b9f5af116d - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:21.748 4076 3051113161128 MainThread INFO nova.compute.manager [req-fc127039-e359-4212-83f6-32b9f5af116d - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:22.638 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '811450F9-F1F9-4713-835B-BE37E6B8BD6D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15057), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:22.779 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:22.779 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:22.779 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:22.779 4076 3051114311752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:22.779 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '811450F9-F1F9-4713-835B-BE37E6B8BD6D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111857), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:22.779 4076 3051114314568 MainThread DEBUG os_win._utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.838342905044556 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:22.816 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-b94db107-a65c-46f4-b936-4bf1b1d4c53f - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:22.816 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-b94db107-a65c-46f4-b936-4bf1b1d4c53f - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:22.856 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-b94db107-a65c-46f4-b936-4bf1b1d4c53f - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:22.891 4076 3051113161128 MainThread INFO nova.compute.manager [req-b94db107-a65c-46f4-b936-4bf1b1d4c53f - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:23.696 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Successfully created port: 7f23350c-75ed-4892-ad64-29eab9820ae5 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:06:23.813 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'EF3B29DD-F35F-4277-9AB1-01AC70347994', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15354), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:23.954 4076 3051093462536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:23.954 4076 3051093462536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:23.954 4076 3051114311752 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:23.954 4076 3051114311752 MainThread INFO nova.compute.manager [-] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] VM Stopped (Lifecycle Event) 2021-09-10 17:06:23.954 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006a\' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)instance-0000006a150623E0-5738-466F-9AF0-B08D962679F6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EF3B29DD-F35F-4277-9AB1-01AC70347994', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109577), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\nNot enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)\n\n'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6)", 'ErrorSummaryDescription': "'instance-0000006a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:23.954 4076 3051114314568 MainThread ERROR nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Failed to change vm state of instance-0000006a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. 2021-09-10 17:06:23.954 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Got request to destroy instance 2021-09-10 17:06:23.981 4076 3051114311752 MainThread DEBUG nova.compute.manager [req-95a655ef-c4c7-43de-881f-ca7355936482 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:23.981 4076 3051114311752 MainThread DEBUG nova.virt.hyperv.vmops [req-95a655ef-c4c7-43de-881f-ca7355936482 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:24.013 4076 3051114311752 MainThread DEBUG nova.compute.manager [req-95a655ef-c4c7-43de-881f-ca7355936482 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:24.077 4076 3051114311752 MainThread INFO nova.compute.manager [req-95a655ef-c4c7-43de-881f-ca7355936482 - - - - -] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:25.779 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:06:25.779 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:25.779 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:25.805 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed vm state of instance-0000006a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:06:25.805 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed state of VM instance-0000006a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:06:25.857 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'CBE2AEB4-6699-4785-8B3F-F36069E119D8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14560), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:25.972 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job succeeded: {'InstanceID': 'CBE2AEB4-6699-4785-8B3F-F36069E119D8', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29675), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:06:25.972 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:25.972 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Removing directory: C:\openstack\instances\instance-0000006a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:06:25.972 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:25.972 4076 3051114314568 MainThread ERROR nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Traceback (most recent call last): 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(*args, **kwargs) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] admin_password, network_info, block_device_info) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(self, *args, **kwds) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] should_plug_vifs=False) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'req_state': req_state}) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] time.sleep(sleep_time) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return f(*args, **kwargs) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return self._wait_for_job(job_path) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] error_desc=err_desc) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During handling of the above exception, another exception occurred: 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Traceback (most recent call last): 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] yield resources 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] accel_info=accel_info) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(*args, **kwargs) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] admin_password, network_info, block_device_info) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(self, *args, **kwds) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] should_plug_vifs=False) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'req_state': req_state}) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] time.sleep(sleep_time) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return f(*args, **kwargs) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return self._wait_for_job(job_path) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] error_desc=err_desc) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:25.972 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:25.972 4076 3051114314568 MainThread INFO nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Terminating instance 2021-09-10 17:06:25.972 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:06:25.972 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Got request to destroy instance 2021-09-10 17:06:25.998 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:06:25.998 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:25.998 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:25.998 4076 3051114314568 MainThread INFO nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:06:25.998 4076 3051114314568 MainThread DEBUG nova.compute.claims [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:06:25.998 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:26.187 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:26.187 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:26.396 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:26.430 4076 3051114314568 MainThread ERROR nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Traceback (most recent call last): 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(*args, **kwargs) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] admin_password, network_info, block_device_info) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(self, *args, **kwds) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] should_plug_vifs=False) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'req_state': req_state}) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] time.sleep(sleep_time) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return f(*args, **kwargs) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return self._wait_for_job(job_path) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] error_desc=err_desc) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] During handling of the above exception, another exception occurred: 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Traceback (most recent call last): 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] accel_info=accel_info) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(*args, **kwargs) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] admin_password, network_info, block_device_info) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return function(self, *args, **kwds) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] should_plug_vifs=False) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'req_state': req_state}) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] time.sleep(sleep_time) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self.force_reraise() 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] raise self.value 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return f(*args, **kwargs) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] return self._wait_for_job(job_path) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] error_desc=err_desc) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. 2021-09-10 17:06:26.430 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.compute.utils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Build of instance fc38b331-efcd-4c62-aecf-3859bffff0bb was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006a' failed to start. Not enough memory in the system to start the virtual machine instance-0000006a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006a' failed to start. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Not enough memory in the system to start the virtual machine instance-0000006a with ram size 1024 megabytes. (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) 'instance-0000006a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 150623E0-5738-466F-9AF0-B08D962679F6) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:06:26.430 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:06:27.201 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Successfully updated port: 7f23350c-75ed-4892-ad64-29eab9820ae5 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:06:27.232 4076 3051093462008 GreenThread-94 DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-a891198b-c066-46f2-bcbd-290a870d60bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:27.232 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:06:27.533 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:06:28.445 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Received event network-changed-7f23350c-75ed-4892-ad64-29eab9820ae5 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:28.445 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Refreshing instance network info cache due to event network-changed-7f23350c-75ed-4892-ad64-29eab9820ae5. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:06:28.687 4076 3051093462008 GreenThread-94 DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Updating instance_info_cache with network_info: [{"id": "7f23350c-75ed-4892-ad64-29eab9820ae5", "address": "fa:16:3e:7d:57:6b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f23350c-75", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:28.748 4076 3051093462008 GreenThread-94 DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-a891198b-c066-46f2-bcbd-290a870d60bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:28.748 4076 3051093462008 GreenThread-94 DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Instance network_info: |[{"id": "7f23350c-75ed-4892-ad64-29eab9820ae5", "address": "fa:16:3e:7d:57:6b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f23350c-75", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:06:28.748 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-a891198b-c066-46f2-bcbd-290a870d60bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:28.748 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Refreshing network info cache for port 7f23350c-75ed-4892-ad64-29eab9820ae5 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:06:28.748 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Preparing to wait for external event network-vif-plugged-7f23350c-75ed-4892-ad64-29eab9820ae5 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:06:28.748 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:28.748 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:28.748 4076 3051093462360 MainThread DEBUG os_win.utils.compute.vmutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-00000068 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:06:28.794 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:28.813 4076 3051114314568 MainThread INFO nova.compute.manager [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: fc38b331-efcd-4c62-aecf-3859bffff0bb] Took 2.38 seconds to deallocate network for instance. 2021-09-10 17:06:29.061 4076 3051114314568 MainThread INFO nova.scheduler.client.report [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Deleted allocations for instance fc38b331-efcd-4c62-aecf-3859bffff0bb 2021-09-10 17:06:29.107 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-2a837ba2-9334-4c35-8b06-a41b8373d5ca 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "fc38b331-efcd-4c62-aecf-3859bffff0bb" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.094s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:29.655 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:06:30.639 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Updated VIF entry in instance network info cache for port 7f23350c-75ed-4892-ad64-29eab9820ae5. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:06:30.639 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Updating instance_info_cache with network_info: [{"id": "7f23350c-75ed-4892-ad64-29eab9820ae5", "address": "fa:16:3e:7d:57:6b", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.2", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7f23350c-75", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:30.653 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-7d3247de-aa7f-4c8b-bca8-c461c5a7b634 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-a891198b-c066-46f2-bcbd-290a870d60bd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:34.809 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:34.809 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:06:34.998 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:34.998 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:06:35.878 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-cf6fe89b-bc97-4bb0-8c45-2ecc9247c5f6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Received event network-vif-plugged-7f23350c-75ed-4892-ad64-29eab9820ae5 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:35.878 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-cf6fe89b-bc97-4bb0-8c45-2ecc9247c5f6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:35.878 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-cf6fe89b-bc97-4bb0-8c45-2ecc9247c5f6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:35.878 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-cf6fe89b-bc97-4bb0-8c45-2ecc9247c5f6 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Processing event network-vif-plugged-7f23350c-75ed-4892-ad64-29eab9820ae5 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:06:35.911 4076 3051093462360 MainThread INFO nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Using config drive for instance 2021-09-10 17:06:36.450 4076 3051093462360 MainThread INFO nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Creating config drive at C:\openstack\instances\instance-00000068\configdrive.iso 2021-09-10 17:06:36.480 4076 3051093462360 MainThread DEBUG oslo_concurrency.processutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000068\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpestyznu7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:06:36.781 4076 3051093462360 MainThread DEBUG oslo_concurrency.processutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000068\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpestyznu7" returned: 0 in 0.296s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:06:37.169 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-00000068: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:06:37.223 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:06:37.254 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '219D0915-94F6-4215-965B-4DCFA97B19E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18722), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:37.382 4076 3051113431464 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:37.382 4076 3051113431464 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:37.394 4076 3051093462008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:37.394 4076 3051093462008 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:37.394 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '219D0915-94F6-4215-965B-4DCFA97B19E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110594), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:37.394 4076 3051093462360 MainThread DEBUG os_win._utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1714158058166504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:37.418 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-9c3da3ae-dfa6-4299-b532-0e2f9c7defaf - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:37.418 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-9c3da3ae-dfa6-4299-b532-0e2f9c7defaf - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:37.478 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-9c3da3ae-dfa6-4299-b532-0e2f9c7defaf - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:37.515 4076 3051093462008 MainThread INFO nova.compute.manager [req-9c3da3ae-dfa6-4299-b532-0e2f9c7defaf - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:37.744 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:37.764 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:37.764 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:06:37.764 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:38.408 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '523D05CD-4224-4202-A2F7-0C48A9C1554C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14938), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:38.546 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:38.546 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:38.546 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:38.546 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:38.560 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '523D05CD-4224-4202-A2F7-0C48A9C1554C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113498), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:38.560 4076 3051093462360 MainThread DEBUG os_win._utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3375828266143799 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:38.590 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-3e459f06-ae41-4c8d-b84b-a3ca29f6b515 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:38.590 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-3e459f06-ae41-4c8d-b84b-a3ca29f6b515 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:38.636 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-3e459f06-ae41-4c8d-b84b-a3ca29f6b515 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:38.653 4076 3051113161128 MainThread INFO nova.compute.manager [req-3e459f06-ae41-4c8d-b84b-a3ca29f6b515 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:38.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:38.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:38.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:06:39.593 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'EF283897-CF6F-4B80-BA02-CC590B4F0FF4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14919), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:39.727 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:39.737 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:39.737 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:39.737 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:39.737 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EF283897-CF6F-4B80-BA02-CC590B4F0FF4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110522), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:39.737 4076 3051093462360 MainThread DEBUG os_win._utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.513883590698242 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:39.784 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8207d2-e98e-4004-99ec-0ad33d7605b0 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:39.784 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8207d2-e98e-4004-99ec-0ad33d7605b0 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:39.810 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:39.840 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8207d2-e98e-4004-99ec-0ad33d7605b0 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:39.887 4076 3051113161128 MainThread INFO nova.compute.manager [req-de8207d2-e98e-4004-99ec-0ad33d7605b0 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:40.770 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '3C7D4349-A2C3-4899-BD6E-2C548936F592', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15274), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:40.794 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:40.889 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:40.889 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:40.912 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:40.912 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:40.912 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3C7D4349-A2C3-4899-BD6E-2C548936F592', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110545), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:40.912 4076 3051093462360 MainThread DEBUG os_win._utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.689594030380249 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:40.934 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-48a40ae6-7d42-458a-b405-9d87a44012e4 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:40.934 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-48a40ae6-7d42-458a-b405-9d87a44012e4 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:40.993 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-48a40ae6-7d42-458a-b405-9d87a44012e4 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:41.028 4076 3051113161128 MainThread INFO nova.compute.manager [req-48a40ae6-7d42-458a-b405-9d87a44012e4 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:41.127 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:41.168 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:06:41.566 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:41.566 4076 3051093462008 MainThread DEBUG nova.virt.hardware [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:06:41.566 4076 3051093462008 MainThread INFO nova.compute.claims [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Claim successful on node n-h1-808474-1 2021-09-10 17:06:41.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:41.904 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:41.904 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:41.955 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '4F09563B-BAD9-4F97-861C-FBE9329421EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15067), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:42.040 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:06:42.068 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:06:42.068 4076 3051093462008 MainThread DEBUG nova.compute.provider_tree [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:06:42.092 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:42.092 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:42.092 4076 3051113431464 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:42.092 4076 3051113431464 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:42.107 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:06:42.107 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4F09563B-BAD9-4F97-861C-FBE9329421EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110910), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:42.107 4076 3051093462360 MainThread DEBUG os_win._utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8839805126190186 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:42.107 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-81e16e9c-c24d-4818-8bd8-859d7f638c11 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:42.107 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-81e16e9c-c24d-4818-8bd8-859d7f638c11 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:42.154 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:06:42.154 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.hostops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:42.171 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-81e16e9c-c24d-4818-8bd8-859d7f638c11 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:42.171 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.hostops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:42.216 4076 3051113161128 MainThread INFO nova.compute.manager [req-81e16e9c-c24d-4818-8bd8-859d7f638c11 - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:42.387 4076 3051093462008 MainThread DEBUG nova.compute.provider_tree [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:42.404 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:42.404 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.843s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:42.404 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:06:42.512 4076 3051093462008 MainThread DEBUG nova.compute.utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:06:42.512 4076 3051113161128 GreenThread-95 DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:06:42.512 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:06:42.543 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:06:42.700 4076 3051113161128 GreenThread-95 DEBUG nova.policy [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5e00875ca35f4501afe5474947cd0708', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89ea24a7b47a4f4a9015423bfcc23f3b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:06:42.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:42.800 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:06:42.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:42.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:42.800 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:06:42.800 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:42.800 4076 3051093462008 MainThread INFO nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Spawning new instance 2021-09-10 17:06:42.821 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:42.821 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:42.821 4076 3051093462008 MainThread DEBUG nova.block_device [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:06:42.821 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:42.821 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:42.832 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:42.832 4076 3051093462008 MainThread DEBUG os_win.utils.pathutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Creating directory: C:\openstack\instances\instance-0000006b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:06:42.832 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:06:42.857 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Resizing VHD C:\openstack\instances\instance-0000006b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:06:43.039 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2186MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:06:43.039 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:43.133 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '1FB6ADB2-8D32-4FAB-87F5-1482D5F22425', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18894), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:43.265 4076 3051113431464 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:43.265 4076 3051113431464 MainThread INFO nova.compute.manager [-] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] VM Stopped (Lifecycle Event) 2021-09-10 17:06:43.265 4076 3051113431992 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:43.282 4076 3051113431992 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:43.282 4076 3051093462360 MainThread ERROR os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000068\' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000068\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)instance-000000687B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1FB6ADB2-8D32-4FAB-87F5-1482D5F22425', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124577), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\nNot enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)\n\n'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB)", 'ErrorSummaryDescription': "'instance-00000068' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000068.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:43.282 4076 3051093462360 MainThread ERROR nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-00000068 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. 2021-09-10 17:06:43.282 4076 3051093462360 MainThread INFO nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Got request to destroy instance 2021-09-10 17:06:43.311 4076 3051113431464 MainThread DEBUG nova.compute.manager [req-82d083e3-6fc6-4119-8c00-2e8f8a09ff3e - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:43.311 4076 3051113431464 MainThread DEBUG nova.virt.hyperv.vmops [req-82d083e3-6fc6-4119-8c00-2e8f8a09ff3e - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:43.372 4076 3051113431464 MainThread DEBUG nova.compute.manager [req-82d083e3-6fc6-4119-8c00-2e8f8a09ff3e - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:43.430 4076 3051113431464 MainThread INFO nova.compute.manager [req-82d083e3-6fc6-4119-8c00-2e8f8a09ff3e - - - - -] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:43.435 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.435 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a891198b-c066-46f2-bcbd-290a870d60bd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 14 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:06:43.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=14848MB phys_disk=79GB used_disk=14GB total_vcpus=6 used_vcpus=14 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:06:43.502 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:43.523 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:43.739 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:43.739 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:43.739 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:06:43.739 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.688s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:43.843 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Successfully created port: 7a27774f-5489-4105-9a38-976446800bca _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:06:44.731 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:44.731 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:06:45.120 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:06:45.137 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:45.137 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:45.146 4076 3051093462360 MainThread DEBUG os_win.utils.compute.vmutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-00000068 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:06:45.146 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-00000068 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:06:45.197 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '8F522A7F-084F-442E-B6CC-71CDB437A617', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16702), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:45.313 4076 3051093462360 MainThread DEBUG os_win.utils.jobutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': '8F522A7F-084F-442E-B6CC-71CDB437A617', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30243), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:06:45.313 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:45.313 4076 3051093462360 MainThread DEBUG os_win.utils.pathutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-00000068 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:06:45.313 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:45.313 4076 3051093462360 MainThread ERROR nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Traceback (most recent call last): 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(*args, **kwargs) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] admin_password, network_info, block_device_info) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(self, *args, **kwds) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] should_plug_vifs=False) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'req_state': req_state}) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] time.sleep(sleep_time) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return f(*args, **kwargs) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return self._wait_for_job(job_path) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] error_desc=err_desc) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During handling of the above exception, another exception occurred: 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Traceback (most recent call last): 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] yield resources 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] accel_info=accel_info) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(*args, **kwargs) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] admin_password, network_info, block_device_info) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(self, *args, **kwds) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] should_plug_vifs=False) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'req_state': req_state}) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] time.sleep(sleep_time) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return f(*args, **kwargs) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return self._wait_for_job(job_path) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] error_desc=err_desc) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.313 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.313 4076 3051093462360 MainThread INFO nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Terminating instance 2021-09-10 17:06:45.313 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:06:45.313 4076 3051093462360 MainThread INFO nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Got request to destroy instance 2021-09-10 17:06:45.337 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.vmops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:06:45.337 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:45.337 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-00000068" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:45.337 4076 3051093462360 MainThread INFO nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:06:45.337 4076 3051093462360 MainThread DEBUG nova.compute.claims [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:06:45.337 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:45.512 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.hostops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:45.529 4076 3051093462360 MainThread DEBUG nova.virt.hyperv.hostops [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:45.724 4076 3051093462360 MainThread DEBUG nova.compute.provider_tree [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.scheduler.client.report [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:45.747 4076 3051093462360 MainThread ERROR nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Traceback (most recent call last): 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(*args, **kwargs) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] admin_password, network_info, block_device_info) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(self, *args, **kwds) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] should_plug_vifs=False) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'req_state': req_state}) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] time.sleep(sleep_time) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return f(*args, **kwargs) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return self._wait_for_job(job_path) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] error_desc=err_desc) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] During handling of the above exception, another exception occurred: 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Traceback (most recent call last): 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] accel_info=accel_info) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(*args, **kwargs) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] admin_password, network_info, block_device_info) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return function(self, *args, **kwds) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] should_plug_vifs=False) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'req_state': req_state}) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] time.sleep(sleep_time) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self.force_reraise() 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] raise self.value 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return f(*args, **kwargs) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] return self._wait_for_job(job_path) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] error_desc=err_desc) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. 2021-09-10 17:06:45.747 4076 3051093462360 MainThread TRACE nova.compute.manager [instance: a891198b-c066-46f2-bcbd-290a870d60bd] 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.compute.utils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Build of instance a891198b-c066-46f2-bcbd-290a870d60bd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000068' failed to start. Not enough memory in the system to start the virtual machine instance-00000068. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000068' failed to start. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Not enough memory in the system to start the virtual machine instance-00000068 with ram size 1024 megabytes. (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) 'instance-00000068' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7B4A6B2F-0E01-49C5-9B9F-C890BA2E41DB) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:06:45.747 4076 3051093462360 MainThread DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:06:46.368 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Successfully updated port: 7a27774f-5489-4105-9a38-976446800bca _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:06:46.379 4076 3051113161128 GreenThread-95 DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Acquired lock "refresh_cache-1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:46.379 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:06:46.486 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:06:46.790 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:47.039 4076 3051113161128 GreenThread-95 DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Updating instance_info_cache with network_info: [{"id": "7a27774f-5489-4105-9a38-976446800bca", "address": "fa:16:3e:6c:97:e4", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7a27774f-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:47.055 4076 3051113161128 GreenThread-95 DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Releasing lock "refresh_cache-1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:47.055 4076 3051113161128 GreenThread-95 DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Instance network_info: |[{"id": "7a27774f-5489-4105-9a38-976446800bca", "address": "fa:16:3e:6c:97:e4", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7a27774f-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:06:47.055 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Preparing to wait for external event network-vif-plugged-7a27774f-5489-4105-9a38-976446800bca prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:06:47.055 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:47.055 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:47.055 4076 3051093462008 MainThread DEBUG os_win.utils.compute.vmutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Creating VM instance-0000006b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:06:47.172 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Received event network-changed-7a27774f-5489-4105-9a38-976446800bca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:47.172 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Refreshing instance network info cache due to event network-changed-7a27774f-5489-4105-9a38-976446800bca. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:06:47.172 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:06:47.172 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Refreshing network info cache for port 7a27774f-5489-4105-9a38-976446800bca _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:06:47.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:06:47.778 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:06:47.778 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:06:47.936 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:06:48.044 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Updated VIF entry in instance network info cache for port 7a27774f-5489-4105-9a38-976446800bca. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:06:48.044 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Updating instance_info_cache with network_info: [{"id": "7a27774f-5489-4105-9a38-976446800bca", "address": "fa:16:3e:6c:97:e4", "network": {"id": "d1d0dbf5-f2ce-4eeb-bc38-d716096517ef", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-413735321-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "89ea24a7b47a4f4a9015423bfcc23f3b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap7a27774f-54", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:48.092 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-fe002341-be5c-45dd-8a1b-5dc8247986fc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:06:48.836 4076 3051093462360 MainThread DEBUG nova.network.neutron [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:06:48.883 4076 3051093462360 MainThread INFO nova.compute.manager [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: a891198b-c066-46f2-bcbd-290a870d60bd] Took 3.14 seconds to deallocate network for instance. 2021-09-10 17:06:49.092 4076 3051093462360 MainThread INFO nova.scheduler.client.report [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance a891198b-c066-46f2-bcbd-290a870d60bd 2021-09-10 17:06:49.148 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [req-ae042bb7-1b30-4ba1-b9c5-e77425baf409 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "a891198b-c066-46f2-bcbd-290a870d60bd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.046s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:54.034 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-2351de95-a8bf-48f7-aba6-595b3c4b49e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Received event network-vif-plugged-7a27774f-5489-4105-9a38-976446800bca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:06:54.034 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-2351de95-a8bf-48f7-aba6-595b3c4b49e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:54.034 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-2351de95-a8bf-48f7-aba6-595b3c4b49e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:54.034 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-2351de95-a8bf-48f7-aba6-595b3c4b49e8 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Processing event network-vif-plugged-7a27774f-5489-4105-9a38-976446800bca _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:06:54.070 4076 3051093462008 MainThread INFO nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Using config drive for instance 2021-09-10 17:06:54.711 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:54.750 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:06:54.939 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:54.939 4076 3051114314568 MainThread DEBUG nova.virt.hardware [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:06:54.939 4076 3051114314568 MainThread INFO nova.compute.claims [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Claim successful on node n-h1-808474-1 2021-09-10 17:06:55.008 4076 3051093462008 MainThread INFO nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Creating config drive at C:\openstack\instances\instance-0000006b\configdrive.iso 2021-09-10 17:06:55.055 4076 3051093462008 MainThread DEBUG oslo_concurrency.processutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpsi6istbt execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:06:55.266 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:06:55.280 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:06:55.395 4076 3051093462008 MainThread DEBUG oslo_concurrency.processutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpsi6istbt" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:06:55.505 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:06:55.532 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:06:55.532 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.593s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:55.532 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:06:55.650 4076 3051114314568 MainThread DEBUG nova.compute.utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:06:55.650 4076 3051113161128 GreenThread-96 DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:06:55.650 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:06:55.682 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:06:55.797 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Setting boot order for instance: instance-0000006b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:06:55.847 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:06:55.847 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:06:55.847 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Spawning new instance 2021-09-10 17:06:55.872 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:55.872 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:55.872 4076 3051114314568 MainThread DEBUG nova.block_device [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:06:55.872 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:55.872 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:55.872 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '632714C1-3811-4929-AFC7-D131A1D1ABFA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16091), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:55.888 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating directory: C:\openstack\instances\instance-0000006c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:06:55.888 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:06:55.919 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Resizing VHD C:\openstack\instances\instance-0000006c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:06:56.021 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:56.021 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:56.021 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:56.021 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:06:56.021 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '632714C1-3811-4929-AFC7-D131A1D1ABFA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120957), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 55), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 55), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:56.021 4076 3051093462008 MainThread DEBUG os_win._utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17348098754882812 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:56.053 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-dd71f9d4-5b33-4852-b5e9-89f6f3546e92 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:56.053 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-dd71f9d4-5b33-4852-b5e9-89f6f3546e92 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:56.108 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-dd71f9d4-5b33-4852-b5e9-89f6f3546e92 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:56.141 4076 3051114314920 MainThread INFO nova.compute.manager [req-dd71f9d4-5b33-4852-b5e9-89f6f3546e92 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:56.520 4076 3051113161128 GreenThread-96 DEBUG nova.policy [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd3b19a6ed8024fe7b99f5aa4978b5af3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'eecab78b34b0468f8c42e4b7f3444918', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:06:57.060 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '19284602-2E91-40C9-BF9E-AE16927419C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15399), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:57.207 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:57.207 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:57.207 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:57.207 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:06:57.207 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '19284602-2E91-40C9-BF9E-AE16927419C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118922), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 57), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:57.216 4076 3051093462008 MainThread DEBUG os_win._utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.368730068206787 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:57.216 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-82c5c567-76ea-461f-9422-9d163acbd082 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:57.216 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-82c5c567-76ea-461f-9422-9d163acbd082 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:57.266 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-82c5c567-76ea-461f-9422-9d163acbd082 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:57.309 4076 3051114314920 MainThread INFO nova.compute.manager [req-82c5c567-76ea-461f-9422-9d163acbd082 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:58.236 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '45094ED1-FBED-45A0-82C9-FFEA66ED559E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14857), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:58.376 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:58.376 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:58.376 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:58.376 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:06:58.376 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '45094ED1-FBED-45A0-82C9-FFEA66ED559E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113178), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:58.376 4076 3051093462008 MainThread DEBUG os_win._utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5289251804351807 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:58.418 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-2e0f1b1a-c9e7-4517-89f3-8b0f0b2a7114 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:58.418 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-2e0f1b1a-c9e7-4517-89f3-8b0f0b2a7114 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:58.467 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-2e0f1b1a-c9e7-4517-89f3-8b0f0b2a7114 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:58.524 4076 3051114314920 MainThread INFO nova.compute.manager [req-2e0f1b1a-c9e7-4517-89f3-8b0f0b2a7114 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:06:59.391 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '9E00EC63-FFFC-4D83-8C8D-F279E2238699', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14845), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:06:59.466 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Successfully created port: b6f00d5a-4f80-4da8-978b-996fe345c0da _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:06:59.522 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:06:59.522 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:06:59.533 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:06:59.533 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:06:59.533 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9E00EC63-FFFC-4D83-8C8D-F279E2238699', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111487), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 6, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 6, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:06:59.533 4076 3051093462008 MainThread DEBUG os_win._utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6857290267944336 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:06:59.554 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-d5c8db73-4504-421d-999d-16a0e31da91c - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:06:59.554 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-d5c8db73-4504-421d-999d-16a0e31da91c - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:06:59.595 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-d5c8db73-4504-421d-999d-16a0e31da91c - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:06:59.640 4076 3051114314920 MainThread INFO nova.compute.manager [req-d5c8db73-4504-421d-999d-16a0e31da91c - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:00.550 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '57C75684-E634-4D75-BCCB-9AFC4234AB32', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15093), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:00.692 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:00.702 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:00.702 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:00.702 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:07:00.702 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '57C75684-E634-4D75-BCCB-9AFC4234AB32', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110881), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:00.702 4076 3051093462008 MainThread DEBUG os_win._utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.854478120803833 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:00.762 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-519cf5c6-bed8-41d1-b584-d93ff3b5dbe5 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:00.762 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-519cf5c6-bed8-41d1-b584-d93ff3b5dbe5 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:00.794 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-519cf5c6-bed8-41d1-b584-d93ff3b5dbe5 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:00.876 4076 3051114314920 MainThread INFO nova.compute.manager [req-519cf5c6-bed8-41d1-b584-d93ff3b5dbe5 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:01.726 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '656C5785-84FB-4844-A304-7FE55EF45C52', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16164), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:01.871 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:01.871 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] VM Stopped (Lifecycle Event) 2021-09-10 17:07:01.871 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:01.871 4076 3051093462360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:01.871 4076 3051093462008 MainThread ERROR os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006b\' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)instance-0000006bF0680951-993F-4AA6-81E2-299BAB1ABFA1%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '656C5785-84FB-4844-A304-7FE55EF45C52', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120156), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\nNot enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)\n\n'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1)", 'ErrorSummaryDescription': "'instance-0000006b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:01.871 4076 3051093462008 MainThread ERROR nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Failed to change vm state of instance-0000006b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. 2021-09-10 17:07:01.871 4076 3051093462008 MainThread INFO nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Got request to destroy instance 2021-09-10 17:07:01.930 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-6f9c2150-ec15-4091-86f0-60c8bfef06e0 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:01.930 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-6f9c2150-ec15-4091-86f0-60c8bfef06e0 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:01.985 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-6f9c2150-ec15-4091-86f0-60c8bfef06e0 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:02.023 4076 3051114314920 MainThread INFO nova.compute.manager [req-6f9c2150-ec15-4091-86f0-60c8bfef06e0 - - - - -] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:02.622 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Successfully updated port: b6f00d5a-4f80-4da8-978b-996fe345c0da _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:07:02.655 4076 3051113161128 GreenThread-96 DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Acquired lock "refresh_cache-1a16a862-5759-43b6-88bb-d0ce8a7434da" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:07:02.655 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:07:03.152 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:07:03.409 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Received event network-changed-b6f00d5a-4f80-4da8-978b-996fe345c0da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:07:03.409 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Refreshing instance network info cache due to event network-changed-b6f00d5a-4f80-4da8-978b-996fe345c0da. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:07:03.730 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:07:03.733 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:03.733 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:03.733 4076 3051093462008 MainThread DEBUG os_win.utils.compute.vmutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Successfully changed vm state of instance-0000006b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:07:03.733 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Successfully changed state of VM instance-0000006b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:07:03.793 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Waiting for WMI job: {'InstanceID': '96227211-A0DB-44B6-B7B4-3E79F9E66F45', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14585), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:03.914 4076 3051093462008 MainThread DEBUG os_win.utils.jobutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] WMI job succeeded: {'InstanceID': '96227211-A0DB-44B6-B7B4-3E79F9E66F45', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28692), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:07:03.914 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:03.914 4076 3051093462008 MainThread DEBUG os_win.utils.pathutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Removing directory: C:\openstack\instances\instance-0000006b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:07:03.914 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:03.914 4076 3051093462008 MainThread ERROR nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Traceback (most recent call last): 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(*args, **kwargs) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] admin_password, network_info, block_device_info) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(self, *args, **kwds) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] should_plug_vifs=False) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'req_state': req_state}) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] time.sleep(sleep_time) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return f(*args, **kwargs) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return self._wait_for_job(job_path) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] error_desc=err_desc) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During handling of the above exception, another exception occurred: 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Traceback (most recent call last): 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] yield resources 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] accel_info=accel_info) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(*args, **kwargs) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] admin_password, network_info, block_device_info) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(self, *args, **kwds) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] should_plug_vifs=False) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'req_state': req_state}) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] time.sleep(sleep_time) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return f(*args, **kwargs) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return self._wait_for_job(job_path) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] error_desc=err_desc) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:03.914 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:03.914 4076 3051093462008 MainThread INFO nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Terminating instance 2021-09-10 17:07:03.914 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:07:03.914 4076 3051093462008 MainThread INFO nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Got request to destroy instance 2021-09-10 17:07:03.936 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:07:03.936 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:03.936 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "instance-0000006b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:03.936 4076 3051093462008 MainThread INFO nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:07:03.936 4076 3051093462008 MainThread DEBUG nova.compute.claims [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:07:03.936 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:04.152 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.hostops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:04.154 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.hostops [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:04.233 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:04.287 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:07:04.371 4076 3051093462008 MainThread DEBUG nova.compute.provider_tree [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:07:04.399 4076 3051093462008 MainThread DEBUG nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:07:04.399 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:04.399 4076 3051093462008 MainThread ERROR nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Traceback (most recent call last): 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(*args, **kwargs) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] admin_password, network_info, block_device_info) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(self, *args, **kwds) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] should_plug_vifs=False) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'req_state': req_state}) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] time.sleep(sleep_time) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return f(*args, **kwargs) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return self._wait_for_job(job_path) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] error_desc=err_desc) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] During handling of the above exception, another exception occurred: 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Traceback (most recent call last): 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] accel_info=accel_info) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(*args, **kwargs) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] admin_password, network_info, block_device_info) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return function(self, *args, **kwds) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] should_plug_vifs=False) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'req_state': req_state}) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] time.sleep(sleep_time) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self.force_reraise() 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] raise self.value 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return f(*args, **kwargs) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] return self._wait_for_job(job_path) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] error_desc=err_desc) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. 2021-09-10 17:07:04.399 4076 3051093462008 MainThread TRACE nova.compute.manager [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] 2021-09-10 17:07:04.399 4076 3051093462008 MainThread DEBUG nova.compute.utils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:07:04.406 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Build of instance 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006b' failed to start. Not enough memory in the system to start the virtual machine instance-0000006b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006b' failed to start. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Not enough memory in the system to start the virtual machine instance-0000006b with ram size 1024 megabytes. (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) 'instance-0000006b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F0680951-993F-4AA6-81E2-299BAB1ABFA1) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:07:04.406 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:07:04.406 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:07:04.406 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:07:04.406 4076 3051093462008 MainThread DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:07:04.597 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:04.597 4076 3051093462184 MainThread DEBUG nova.virt.hardware [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:07:04.597 4076 3051093462184 MainThread INFO nova.compute.claims [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Claim successful on node n-h1-808474-1 2021-09-10 17:07:04.833 4076 3051113161128 GreenThread-96 DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Updating instance_info_cache with network_info: [{"id": "b6f00d5a-4f80-4da8-978b-996fe345c0da", "address": "fa:16:3e:04:09:6d", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb6f00d5a-4f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:04.842 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.hostops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:04.868 4076 3051113161128 GreenThread-96 DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Releasing lock "refresh_cache-1a16a862-5759-43b6-88bb-d0ce8a7434da" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:07:04.868 4076 3051113161128 GreenThread-96 DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Instance network_info: |[{"id": "b6f00d5a-4f80-4da8-978b-996fe345c0da", "address": "fa:16:3e:04:09:6d", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb6f00d5a-4f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:07:04.868 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-1a16a862-5759-43b6-88bb-d0ce8a7434da" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:07:04.868 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Refreshing network info cache for port b6f00d5a-4f80-4da8-978b-996fe345c0da _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:07:04.868 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Preparing to wait for external event network-vif-plugged-b6f00d5a-4f80-4da8-978b-996fe345c0da prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:07:04.868 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:04.868 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:04.868 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Creating VM instance-0000006c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:07:04.883 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.hostops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:05.074 4076 3051093462184 MainThread DEBUG nova.compute.provider_tree [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:07:05.117 4076 3051093462184 MainThread DEBUG nova.scheduler.client.report [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:07:05.117 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:05.117 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:07:05.244 4076 3051093462184 MainThread DEBUG nova.compute.utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:07:05.254 4076 3051114313864 GreenThread-97 DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:07:05.254 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:07:05.282 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:07:05.545 4076 3051114313864 GreenThread-97 DEBUG nova.policy [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6da982fe4daa44b998cd4ae6900212fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0928f57ccc9245e28c79dd9cce7e395b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:07:05.605 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:07:05.618 4076 3051093462184 MainThread INFO nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Spawning new instance 2021-09-10 17:07:05.618 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:05.618 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:05.618 4076 3051093462184 MainThread DEBUG nova.block_device [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:07:05.636 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:05.636 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:05.640 4076 3051093462184 MainThread DEBUG os_win.utils.pathutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating directory: C:\openstack\instances\instance-0000006d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:07:05.640 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:07:05.654 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Resizing VHD C:\openstack\instances\instance-0000006d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:07:05.761 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:07:06.204 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Updated VIF entry in instance network info cache for port b6f00d5a-4f80-4da8-978b-996fe345c0da. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:07:06.204 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Updating instance_info_cache with network_info: [{"id": "b6f00d5a-4f80-4da8-978b-996fe345c0da", "address": "fa:16:3e:04:09:6d", "network": {"id": "8a0fc2b4-bff6-4887-a969-81fb448d568f", "bridge": null, "label": "tempest-DeleteServersTestJSON-1414296899-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "eecab78b34b0468f8c42e4b7f3444918", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb6f00d5a-4f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:06.246 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-a818cc93-2763-44a1-985c-0fec4261c74f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-1a16a862-5759-43b6-88bb-d0ce8a7434da" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:07:07.251 4076 3051093462008 MainThread DEBUG nova.network.neutron [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:07.320 4076 3051093462008 MainThread INFO nova.compute.manager [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] [instance: 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a] Took 2.92 seconds to deallocate network for instance. 2021-09-10 17:07:07.732 4076 3051093462008 MainThread INFO nova.scheduler.client.report [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Deleted allocations for instance 1e1ed6b3-ce50-4c81-bc0f-08b715541c7a 2021-09-10 17:07:07.779 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-7fab1e2d-1ec8-46b6-9074-201b562b5ab0 5e00875ca35f4501afe5474947cd0708 89ea24a7b47a4f4a9015423bfcc23f3b - default default] Lock "1e1ed6b3-ce50-4c81-bc0f-08b715541c7a" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 26.656s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:08.824 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Successfully created port: 2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:07:12.060 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Successfully updated port: 2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:07:12.100 4076 3051114313864 GreenThread-97 DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Acquired lock "refresh_cache-8ef051ab-9f0f-4693-a09a-0b85b471b851" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:07:12.100 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:07:12.310 4076 3051113433224 MainThread DEBUG nova.compute.manager [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Received event network-changed-2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:07:12.310 4076 3051113433224 MainThread DEBUG nova.compute.manager [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Refreshing instance network info cache due to event network-changed-2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:07:12.398 4076 3051113433048 MainThread DEBUG nova.compute.manager [req-c2e41684-ca82-469e-9b4b-2aa7ad358b95 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Received event network-vif-plugged-b6f00d5a-4f80-4da8-978b-996fe345c0da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:07:12.398 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [req-c2e41684-ca82-469e-9b4b-2aa7ad358b95 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:12.398 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [req-c2e41684-ca82-469e-9b4b-2aa7ad358b95 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:12.398 4076 3051113433048 MainThread DEBUG nova.compute.manager [req-c2e41684-ca82-469e-9b4b-2aa7ad358b95 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Processing event network-vif-plugged-b6f00d5a-4f80-4da8-978b-996fe345c0da _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:07:12.465 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Using config drive for instance 2021-09-10 17:07:12.481 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:07:13.332 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Creating config drive at C:\openstack\instances\instance-0000006c\configdrive.iso 2021-09-10 17:07:13.367 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp355x5s49 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:07:13.684 4076 3051114314568 MainThread DEBUG oslo_concurrency.processutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp355x5s49" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:07:13.771 4076 3051114313864 GreenThread-97 DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Updating instance_info_cache with network_info: [{"id": "2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f", "address": "fa:16:3e:48:53:cc", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2bfe99d5-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:13.793 4076 3051114313864 GreenThread-97 DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Releasing lock "refresh_cache-8ef051ab-9f0f-4693-a09a-0b85b471b851" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:07:13.793 4076 3051114313864 GreenThread-97 DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Instance network_info: |[{"id": "2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f", "address": "fa:16:3e:48:53:cc", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2bfe99d5-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:07:13.793 4076 3051113433224 MainThread DEBUG oslo_concurrency.lockutils [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-8ef051ab-9f0f-4693-a09a-0b85b471b851" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:07:13.793 4076 3051113433224 MainThread DEBUG nova.network.neutron [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Refreshing network info cache for port 2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:07:13.793 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Preparing to wait for external event network-vif-plugged-2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:07:13.793 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:13.793 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:13.793 4076 3051093462184 MainThread DEBUG os_win.utils.compute.vmutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Creating VM instance-0000006d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:07:14.068 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Setting boot order for instance: instance-0000006c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:07:14.121 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:07:14.145 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '7D322853-DF1C-41E0-8EF2-028B875050A9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15198), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:14.285 4076 3051093462008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:14.294 4076 3051093462008 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:14.294 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:14.294 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:14.294 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7D322853-DF1C-41E0-8EF2-028B875050A9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113296), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:14.294 4076 3051114314568 MainThread DEBUG os_win._utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17262053489685059 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:14.329 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-e52935f0-7446-434a-855e-295b2cc4dcb9 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:14.329 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-e52935f0-7446-434a-855e-295b2cc4dcb9 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:14.356 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-e52935f0-7446-434a-855e-295b2cc4dcb9 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:14.430 4076 3051093462008 MainThread INFO nova.compute.manager [req-e52935f0-7446-434a-855e-295b2cc4dcb9 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:14.746 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:07:15.327 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '0284704D-BB3B-4A97-822C-33A07CDEFCB3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14541), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:15.464 4076 3051114313864 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:15.470 4076 3051114313864 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:15.470 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:15.470 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:15.470 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0284704D-BB3B-4A97-822C-33A07CDEFCB3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111190), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:15.470 4076 3051114314568 MainThread DEBUG os_win._utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3491015434265137 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:15.508 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-20ec25db-2e99-4561-bd8b-95ea845c753f - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:15.508 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-20ec25db-2e99-4561-bd8b-95ea845c753f - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:15.554 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-20ec25db-2e99-4561-bd8b-95ea845c753f - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:15.567 4076 3051114313864 MainThread INFO nova.compute.manager [req-20ec25db-2e99-4561-bd8b-95ea845c753f - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:16.497 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'DF01DCDA-F8D5-4301-8183-929C6C5C35EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16067), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:16.631 4076 3051114313864 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:16.631 4076 3051114313864 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:16.631 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:16.643 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:16.643 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DF01DCDA-F8D5-4301-8183-929C6C5C35EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111140), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:16.643 4076 3051114314568 MainThread DEBUG os_win._utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.521697521209717 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:16.669 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-161a7e62-cc4b-4c5c-8bdc-916994bb1ac4 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:16.669 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-161a7e62-cc4b-4c5c-8bdc-916994bb1ac4 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:16.716 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-161a7e62-cc4b-4c5c-8bdc-916994bb1ac4 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:16.752 4076 3051114313864 MainThread INFO nova.compute.manager [req-161a7e62-cc4b-4c5c-8bdc-916994bb1ac4 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:16.996 4076 3051113433224 MainThread DEBUG nova.network.neutron [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Updated VIF entry in instance network info cache for port 2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:07:16.996 4076 3051113433224 MainThread DEBUG nova.network.neutron [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Updating instance_info_cache with network_info: [{"id": "2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f", "address": "fa:16:3e:48:53:cc", "network": {"id": "f84be94a-cbca-4401-b34e-c0e01bb632c9", "bridge": null, "label": "tempest-AttachVolumeNegativeTest-2088231271-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "0928f57ccc9245e28c79dd9cce7e395b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2bfe99d5-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:17.027 4076 3051113433224 MainThread DEBUG oslo_concurrency.lockutils [req-1895e668-62ed-4ca5-9920-457b93f48a57 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-8ef051ab-9f0f-4693-a09a-0b85b471b851" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:07:17.673 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '78621ADD-521D-4CD9-9A8B-648662AC2EC4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15388), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:17.801 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '78621ADD-521D-4CD9-9A8B-648662AC2EC4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120693), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:17.816 4076 3051114314568 MainThread DEBUG os_win._utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.695396900177002 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:17.816 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:17.831 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:17.831 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:17.831 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:17.859 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-840e100f-8e19-4e80-9ef1-de93e395acff - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:17.859 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-840e100f-8e19-4e80-9ef1-de93e395acff - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:17.903 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-840e100f-8e19-4e80-9ef1-de93e395acff - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:17.927 4076 3051114314920 MainThread INFO nova.compute.manager [req-840e100f-8e19-4e80-9ef1-de93e395acff - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:18.829 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '0E595C07-A425-45A4-848B-F04A26F6844E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14300), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:18.967 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:18.967 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:18.967 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:18.967 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:18.967 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0E595C07-A425-45A4-848B-F04A26F6844E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109625), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:18.967 4076 3051114314568 MainThread DEBUG os_win._utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.84568190574646 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:18.999 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-d0a1798e-8164-4efb-ac02-608722b507d7 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:18.999 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-d0a1798e-8164-4efb-ac02-608722b507d7 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:19.027 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-d0a1798e-8164-4efb-ac02-608722b507d7 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:19.075 4076 3051114314920 MainThread INFO nova.compute.manager [req-d0a1798e-8164-4efb-ac02-608722b507d7 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:19.981 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': 'D7678FB0-B9B3-4B54-B946-F60D6A57CFE4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14971), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:20.132 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:20.132 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] VM Stopped (Lifecycle Event) 2021-09-10 17:07:20.132 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:20.132 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:20.171 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-b92622d0-8611-4b51-9192-717a794c0457 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:20.171 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-b92622d0-8611-4b51-9192-717a794c0457 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:20.209 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-b92622d0-8611-4b51-9192-717a794c0457 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:20.249 4076 3051114314920 MainThread INFO nova.compute.manager [req-b92622d0-8611-4b51-9192-717a794c0457 - - - - -] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:20.249 4076 3051114314568 MainThread ERROR os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006c\' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)instance-0000006cA25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D7678FB0-B9B3-4B54-B946-F60D6A57CFE4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117212), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\nNot enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)\n\n'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B)", 'ErrorSummaryDescription': "'instance-0000006c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:20.249 4076 3051114314568 MainThread ERROR nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Failed to change vm state of instance-0000006c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. 2021-09-10 17:07:20.249 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Got request to destroy instance 2021-09-10 17:07:22.077 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:07:22.077 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:22.077 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:22.077 4076 3051114314568 MainThread DEBUG os_win.utils.compute.vmutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed vm state of instance-0000006c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:07:22.077 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Successfully changed state of VM instance-0000006c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:07:22.106 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-14ea12ef-aea0-4a66-a730-d721265b3e5a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Received event network-vif-plugged-2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:07:22.106 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-14ea12ef-aea0-4a66-a730-d721265b3e5a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:22.106 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-14ea12ef-aea0-4a66-a730-d721265b3e5a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:22.106 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-14ea12ef-aea0-4a66-a730-d721265b3e5a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Processing event network-vif-plugged-2bfe99d5-f5bf-47fb-8e91-fff9bfa0eb4f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:07:22.145 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Waiting for WMI job: {'InstanceID': '87E65AF7-6446-4C7E-8E20-FCE4A6FACFBF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18300), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:22.145 4076 3051093462184 MainThread INFO nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Using config drive for instance 2021-09-10 17:07:22.273 4076 3051114314568 MainThread DEBUG os_win.utils.jobutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] WMI job succeeded: {'InstanceID': '87E65AF7-6446-4C7E-8E20-FCE4A6FACFBF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29951), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:07:22.273 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:22.273 4076 3051114314568 MainThread DEBUG os_win.utils.pathutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Removing directory: C:\openstack\instances\instance-0000006c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:07:22.273 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:22.273 4076 3051114314568 MainThread ERROR nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Traceback (most recent call last): 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(*args, **kwargs) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] admin_password, network_info, block_device_info) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(self, *args, **kwds) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] should_plug_vifs=False) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'req_state': req_state}) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] time.sleep(sleep_time) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return f(*args, **kwargs) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return self._wait_for_job(job_path) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] error_desc=err_desc) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During handling of the above exception, another exception occurred: 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Traceback (most recent call last): 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] yield resources 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] accel_info=accel_info) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(*args, **kwargs) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] admin_password, network_info, block_device_info) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(self, *args, **kwds) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] should_plug_vifs=False) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'req_state': req_state}) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] time.sleep(sleep_time) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return f(*args, **kwargs) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return self._wait_for_job(job_path) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] error_desc=err_desc) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.273 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.273 4076 3051114314568 MainThread INFO nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Terminating instance 2021-09-10 17:07:22.273 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:07:22.273 4076 3051114314568 MainThread INFO nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Got request to destroy instance 2021-09-10 17:07:22.291 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:07:22.302 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:22.302 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "instance-0000006c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:22.302 4076 3051114314568 MainThread INFO nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:07:22.302 4076 3051114314568 MainThread DEBUG nova.compute.claims [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:07:22.302 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:22.514 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:22.530 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.hostops [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:22.742 4076 3051114314568 MainThread DEBUG nova.compute.provider_tree [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.scheduler.client.report [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:22.758 4076 3051114314568 MainThread ERROR nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Traceback (most recent call last): 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(*args, **kwargs) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] admin_password, network_info, block_device_info) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(self, *args, **kwds) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] should_plug_vifs=False) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'req_state': req_state}) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] time.sleep(sleep_time) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return f(*args, **kwargs) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return self._wait_for_job(job_path) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] error_desc=err_desc) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] During handling of the above exception, another exception occurred: 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Traceback (most recent call last): 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] accel_info=accel_info) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(*args, **kwargs) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] admin_password, network_info, block_device_info) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return function(self, *args, **kwds) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] should_plug_vifs=False) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'req_state': req_state}) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] time.sleep(sleep_time) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self.force_reraise() 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] raise self.value 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return f(*args, **kwargs) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] return self._wait_for_job(job_path) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] error_desc=err_desc) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. 2021-09-10 17:07:22.758 4076 3051114314568 MainThread TRACE nova.compute.manager [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.compute.utils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Build of instance 1a16a862-5759-43b6-88bb-d0ce8a7434da was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006c' failed to start. Not enough memory in the system to start the virtual machine instance-0000006c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006c' failed to start. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Not enough memory in the system to start the virtual machine instance-0000006c with ram size 1024 megabytes. (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) 'instance-0000006c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A25760ED-95C6-4DE7-BE8A-AD5D1D7B3A4B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:07:22.758 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:07:22.859 4076 3051093462184 MainThread INFO nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Creating config drive at C:\openstack\instances\instance-0000006d\configdrive.iso 2021-09-10 17:07:22.871 4076 3051093462184 MainThread DEBUG oslo_concurrency.processutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpf65qwpjj execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:07:23.189 4076 3051093462184 MainThread DEBUG oslo_concurrency.processutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpf65qwpjj" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:07:23.570 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Setting boot order for instance: instance-0000006d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:07:23.618 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:07:23.643 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '012A99B6-B010-4BC7-9328-DF527397AFE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15346), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:23.781 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:23.781 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:23.781 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:23.781 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:23.781 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '012A99B6-B010-4BC7-9328-DF527397AFE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112042), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:23.781 4076 3051093462184 MainThread DEBUG os_win._utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16337919235229492 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:23.848 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-e80343cb-67ad-427a-b06e-5b8f7b5cb2a8 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:23.848 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-e80343cb-67ad-427a-b06e-5b8f7b5cb2a8 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:23.871 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-e80343cb-67ad-427a-b06e-5b8f7b5cb2a8 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:23.965 4076 3051114314920 MainThread INFO nova.compute.manager [req-e80343cb-67ad-427a-b06e-5b8f7b5cb2a8 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:24.795 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'CB4CA5F7-4C8F-466C-A23D-DC5462828423', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15445), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:24.934 4076 3051114314920 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:24.934 4076 3051114314920 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:24.934 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:24.934 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:24.934 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CB4CA5F7-4C8F-466C-A23D-DC5462828423', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111057), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:24.934 4076 3051093462184 MainThread DEBUG os_win._utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.316396713256836 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:24.934 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-ce505e6a-da9a-42b2-a79b-f4fa6965199c - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:24.934 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [req-ce505e6a-da9a-42b2-a79b-f4fa6965199c - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:25.007 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-ce505e6a-da9a-42b2-a79b-f4fa6965199c - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:25.042 4076 3051114314568 MainThread DEBUG nova.network.neutron [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:25.050 4076 3051114314920 MainThread INFO nova.compute.manager [req-ce505e6a-da9a-42b2-a79b-f4fa6965199c - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:25.070 4076 3051114314568 MainThread INFO nova.compute.manager [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] [instance: 1a16a862-5759-43b6-88bb-d0ce8a7434da] Took 2.31 seconds to deallocate network for instance. 2021-09-10 17:07:25.422 4076 3051114314568 MainThread INFO nova.scheduler.client.report [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Deleted allocations for instance 1a16a862-5759-43b6-88bb-d0ce8a7434da 2021-09-10 17:07:25.460 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-6951f32b-9133-4589-86ef-718bb29031d2 d3b19a6ed8024fe7b99f5aa4978b5af3 eecab78b34b0468f8c42e4b7f3444918 - default default] Lock "1a16a862-5759-43b6-88bb-d0ce8a7434da" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.750s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:25.950 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'F322CADE-70DF-45C3-AAD4-B2A6F334E9D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15768), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:26.097 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:26.097 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:26.097 4076 3051114314568 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:26.097 4076 3051114314568 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:26.097 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F322CADE-70DF-45C3-AAD4-B2A6F334E9D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114452), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:26.097 4076 3051093462184 MainThread DEBUG os_win._utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.4784345626831055 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:26.137 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-afa7025d-b15d-42ab-85c9-6bc18528cb5a - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:26.137 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-afa7025d-b15d-42ab-85c9-6bc18528cb5a - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:26.179 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-afa7025d-b15d-42ab-85c9-6bc18528cb5a - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:26.230 4076 3051114314568 MainThread INFO nova.compute.manager [req-afa7025d-b15d-42ab-85c9-6bc18528cb5a - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:27.107 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'C086E324-B0AB-4E7B-9699-79FD74FA6F39', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15321), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:27.242 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:27.242 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:27.242 4076 3051114314568 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:27.242 4076 3051114314568 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:27.252 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C086E324-B0AB-4E7B-9699-79FD74FA6F39', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110082), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:27.252 4076 3051093462184 MainThread DEBUG os_win._utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.633885145187378 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:27.273 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-cab1b500-f158-462f-87f2-9d2caf808b67 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:27.273 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-cab1b500-f158-462f-87f2-9d2caf808b67 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:27.315 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-cab1b500-f158-462f-87f2-9d2caf808b67 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:27.392 4076 3051114314568 MainThread INFO nova.compute.manager [req-cab1b500-f158-462f-87f2-9d2caf808b67 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:28.267 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '02C888CD-954B-416E-A608-66D671930ACC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14968), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:28.388 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:28.388 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:28.388 4076 3051093462008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:28.388 4076 3051093462008 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:28.418 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '02C888CD-954B-416E-A608-66D671930ACC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114867), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:28.418 4076 3051093462184 MainThread DEBUG os_win._utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.7998504638671875 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:07:28.439 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-4e74ac1f-ba67-4f70-acf2-aba3de6b6f78 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:28.439 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-4e74ac1f-ba67-4f70-acf2-aba3de6b6f78 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:28.468 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-4e74ac1f-ba67-4f70-acf2-aba3de6b6f78 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:28.507 4076 3051093462008 MainThread INFO nova.compute.manager [req-4e74ac1f-ba67-4f70-acf2-aba3de6b6f78 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:29.445 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': 'F6307B87-2F2D-4F55-AA2C-1C1337FFD220', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15232), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:29.583 4076 3051093462008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:07:29.583 4076 3051093462008 MainThread INFO nova.compute.manager [-] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] VM Stopped (Lifecycle Event) 2021-09-10 17:07:29.589 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:29.589 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:29.589 4076 3051093462184 MainThread ERROR os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006d\' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)instance-0000006d9CC2D032-11F3-4080-94A9-11C97F22522A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F6307B87-2F2D-4F55-AA2C-1C1337FFD220', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113414), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 7, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 7, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\nNot enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)\n\n'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A)", 'ErrorSummaryDescription': "'instance-0000006d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:07:29.589 4076 3051093462184 MainThread ERROR nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Failed to change vm state of instance-0000006d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. 2021-09-10 17:07:29.589 4076 3051093462184 MainThread INFO nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Got request to destroy instance 2021-09-10 17:07:29.589 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-d373ffda-54ee-4965-9839-43343ccc7037 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:07:29.589 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [req-d373ffda-54ee-4965-9839-43343ccc7037 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:29.654 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-d373ffda-54ee-4965-9839-43343ccc7037 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:07:29.726 4076 3051093462008 MainThread INFO nova.compute.manager [req-d373ffda-54ee-4965-9839-43343ccc7037 - - - - -] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:07:31.418 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:07:31.418 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:31.418 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:31.450 4076 3051093462184 MainThread DEBUG os_win.utils.compute.vmutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed vm state of instance-0000006d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:07:31.450 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Successfully changed state of VM instance-0000006d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:07:31.499 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Waiting for WMI job: {'InstanceID': '9843A56C-271B-4B38-9E14-E409FE5503CF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15759), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:07:31.627 4076 3051093462184 MainThread DEBUG os_win.utils.jobutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] WMI job succeeded: {'InstanceID': '9843A56C-271B-4B38-9E14-E409FE5503CF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29446), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:07:31.627 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:31.627 4076 3051093462184 MainThread DEBUG os_win.utils.pathutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Removing directory: C:\openstack\instances\instance-0000006d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:07:31.627 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:31.627 4076 3051093462184 MainThread ERROR nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Traceback (most recent call last): 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(*args, **kwargs) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] admin_password, network_info, block_device_info) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(self, *args, **kwds) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] should_plug_vifs=False) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'req_state': req_state}) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] time.sleep(sleep_time) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return f(*args, **kwargs) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return self._wait_for_job(job_path) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] error_desc=err_desc) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During handling of the above exception, another exception occurred: 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Traceback (most recent call last): 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] yield resources 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] accel_info=accel_info) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(*args, **kwargs) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] admin_password, network_info, block_device_info) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(self, *args, **kwds) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] should_plug_vifs=False) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'req_state': req_state}) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] time.sleep(sleep_time) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return f(*args, **kwargs) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return self._wait_for_job(job_path) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] error_desc=err_desc) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:31.627 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:31.627 4076 3051093462184 MainThread INFO nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Terminating instance 2021-09-10 17:07:31.627 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:07:31.627 4076 3051093462184 MainThread INFO nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Got request to destroy instance 2021-09-10 17:07:31.652 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:07:31.652 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:31.652 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "instance-0000006d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:31.652 4076 3051093462184 MainThread INFO nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:07:31.652 4076 3051093462184 MainThread DEBUG nova.compute.claims [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:07:31.664 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:31.819 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.hostops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:31.819 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.hostops [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:32.023 4076 3051093462184 MainThread DEBUG nova.compute.provider_tree [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:07:32.042 4076 3051093462184 MainThread DEBUG nova.scheduler.client.report [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:07:32.042 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:32.042 4076 3051093462184 MainThread ERROR nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Traceback (most recent call last): 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(*args, **kwargs) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] admin_password, network_info, block_device_info) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(self, *args, **kwds) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] should_plug_vifs=False) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'req_state': req_state}) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] time.sleep(sleep_time) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return f(*args, **kwargs) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return self._wait_for_job(job_path) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] error_desc=err_desc) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] During handling of the above exception, another exception occurred: 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Traceback (most recent call last): 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] accel_info=accel_info) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(*args, **kwargs) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] admin_password, network_info, block_device_info) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return function(self, *args, **kwds) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] should_plug_vifs=False) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'req_state': req_state}) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] time.sleep(sleep_time) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self.force_reraise() 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] raise self.value 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return f(*args, **kwargs) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] return self._wait_for_job(job_path) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] error_desc=err_desc) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. 2021-09-10 17:07:32.042 4076 3051093462184 MainThread TRACE nova.compute.manager [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.compute.utils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Build of instance 8ef051ab-9f0f-4693-a09a-0b85b471b851 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006d' failed to start. Not enough memory in the system to start the virtual machine instance-0000006d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006d' failed to start. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Not enough memory in the system to start the virtual machine instance-0000006d with ram size 1024 megabytes. (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) 'instance-0000006d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9CC2D032-11F3-4080-94A9-11C97F22522A) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:07:32.060 4076 3051093462184 MainThread DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:07:33.621 4076 3051093462184 MainThread DEBUG nova.network.neutron [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:33.667 4076 3051093462184 MainThread INFO nova.compute.manager [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] [instance: 8ef051ab-9f0f-4693-a09a-0b85b471b851] Took 1.61 seconds to deallocate network for instance. 2021-09-10 17:07:34.003 4076 3051093462184 MainThread INFO nova.scheduler.client.report [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Deleted allocations for instance 8ef051ab-9f0f-4693-a09a-0b85b471b851 2021-09-10 17:07:34.047 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [req-02fb1f48-adc9-47c1-909c-1ba49e17f827 6da982fe4daa44b998cd4ae6900212fd 0928f57ccc9245e28c79dd9cce7e395b - default default] Lock "8ef051ab-9f0f-4693-a09a-0b85b471b851" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.812s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:34.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:34.778 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:07:35.292 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:07:35.292 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:07:37.070 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:07:37.078 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:07:37.078 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:07:37.078 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:41.007 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:41.007 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:41.823 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:41.855 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:43.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:43.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:43.773 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:07:43.980 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:44.060 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.060 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.060 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.060 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:07:44.079 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.079 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.079 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.079 4076 3051114314920 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.079 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.079 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.079 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.079 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.079 4076 3051113433752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433752 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051093462008 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433928 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433048 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051093462184 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051113433224 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433224 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051113433576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433576 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.090 4076 3051113433400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.090 4076 3051113433400 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:07:44.247 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.311 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.235s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.340 4076 3051113433752 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.266s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.340 4076 3051113433224 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.340 4076 3051113433400 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.340 4076 3051093462184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.417 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.433 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.433 4076 3051113433576 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.475 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.488 4076 3051113433048 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.488 4076 3051113433928 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.804 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:07:44.833 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:44.833 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:44.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:07:44.833 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:44.842 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:45.075 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2186MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:07:45.075 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:07:45.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:07:45.347 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:07:45.401 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:07:45.417 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:07:45.617 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:07:45.653 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:07:45.653 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:07:45.653 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.578s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:07:48.614 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:14.234 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:14.273 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:08:14.459 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:14.459 4076 3051114313864 MainThread DEBUG nova.virt.hardware [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:08:14.459 4076 3051114313864 MainThread INFO nova.compute.claims [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Claim successful on node n-h1-808474-1 2021-09-10 17:08:14.651 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.hostops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:14.686 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.hostops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:14.882 4076 3051114313864 MainThread DEBUG nova.compute.provider_tree [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:08:14.916 4076 3051114313864 MainThread DEBUG nova.scheduler.client.report [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:08:14.916 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:14.916 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:08:15.177 4076 3051114313864 MainThread DEBUG nova.compute.utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:08:15.177 4076 3051113161128 GreenThread-98 DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:08:15.185 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:08:15.214 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:08:15.482 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:08:15.482 4076 3051114313864 MainThread INFO nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Spawning new instance 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG nova.block_device [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:15.507 4076 3051114313864 MainThread DEBUG os_win.utils.pathutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Creating directory: C:\openstack\instances\instance-0000006e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:08:15.522 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:08:15.541 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Resizing VHD C:\openstack\instances\instance-0000006e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:08:15.949 4076 3051113161128 GreenThread-98 DEBUG nova.policy [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c63d410ca6d947fcba0acf004916fa12', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '203bf8c8ff8149d18d5c8dee127f7611', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:08:18.110 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Successfully created port: ac2096e1-702f-4db7-8fe1-f30db9f79f04 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:08:20.861 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Successfully updated port: ac2096e1-702f-4db7-8fe1-f30db9f79f04 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:08:20.883 4076 3051113161128 GreenThread-98 DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Acquired lock "refresh_cache-6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:08:20.883 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:08:20.948 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Received event network-changed-ac2096e1-702f-4db7-8fe1-f30db9f79f04 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:08:20.948 4076 3051093462008 MainThread DEBUG nova.compute.manager [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Refreshing instance network info cache due to event network-changed-ac2096e1-702f-4db7-8fe1-f30db9f79f04. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:08:21.104 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:08:22.672 4076 3051113161128 GreenThread-98 DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Updating instance_info_cache with network_info: [{"id": "ac2096e1-702f-4db7-8fe1-f30db9f79f04", "address": "fa:16:3e:19:22:52", "network": {"id": "393fb198-540a-475c-bfd1-6196803512b9", "bridge": null, "label": "tempest-InstanceActionsTestJSON-162631977-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "203bf8c8ff8149d18d5c8dee127f7611", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapac2096e1-70", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:22.683 4076 3051113161128 GreenThread-98 DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Releasing lock "refresh_cache-6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:08:22.713 4076 3051113161128 GreenThread-98 DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Instance network_info: |[{"id": "ac2096e1-702f-4db7-8fe1-f30db9f79f04", "address": "fa:16:3e:19:22:52", "network": {"id": "393fb198-540a-475c-bfd1-6196803512b9", "bridge": null, "label": "tempest-InstanceActionsTestJSON-162631977-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "203bf8c8ff8149d18d5c8dee127f7611", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapac2096e1-70", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:08:22.713 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:08:22.713 4076 3051093462008 MainThread DEBUG nova.network.neutron [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Refreshing network info cache for port ac2096e1-702f-4db7-8fe1-f30db9f79f04 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:08:22.713 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Preparing to wait for external event network-vif-plugged-ac2096e1-702f-4db7-8fe1-f30db9f79f04 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:08:22.713 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:22.719 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:22.719 4076 3051114313864 MainThread DEBUG os_win.utils.compute.vmutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Creating VM instance-0000006e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:08:23.576 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:08:24.697 4076 3051093462008 MainThread DEBUG nova.network.neutron [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Updated VIF entry in instance network info cache for port ac2096e1-702f-4db7-8fe1-f30db9f79f04. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:08:24.697 4076 3051093462008 MainThread DEBUG nova.network.neutron [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Updating instance_info_cache with network_info: [{"id": "ac2096e1-702f-4db7-8fe1-f30db9f79f04", "address": "fa:16:3e:19:22:52", "network": {"id": "393fb198-540a-475c-bfd1-6196803512b9", "bridge": null, "label": "tempest-InstanceActionsTestJSON-162631977-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "203bf8c8ff8149d18d5c8dee127f7611", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapac2096e1-70", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:24.711 4076 3051093462008 MainThread DEBUG oslo_concurrency.lockutils [req-4779af0c-464c-4bad-9a2e-d75319bd5403 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:08:25.854 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:25.886 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:08:26.119 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:26.119 4076 3051113161128 MainThread DEBUG nova.virt.hardware [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:08:26.119 4076 3051113161128 MainThread INFO nova.compute.claims [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Claim successful on node n-h1-808474-1 2021-09-10 17:08:26.290 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:26.299 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:26.515 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:08:26.530 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:08:26.530 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:26.530 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:08:26.633 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:08:26.633 4076 3051093462184 GreenThread-99 DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:08:26.633 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:08:26.667 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:08:26.885 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:08:26.902 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Spawning new instance 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG nova.block_device [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Creating directory: C:\openstack\instances\instance-0000006f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:08:26.902 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000006f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:08:26.932 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Resizing VHD C:\openstack\instances\instance-0000006f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:08:26.966 4076 3051093462184 GreenThread-99 DEBUG nova.policy [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '95e49dab1ffa48cfb40576e8646baad3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ebc509414a41b9890ca144a70c816d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:08:30.040 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Successfully created port: 2dbc6c2e-1121-4016-ab40-496deb947b2a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:08:30.229 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-c682cc1c-d6a3-435d-8d87-05eb01e1f1f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Received event network-vif-plugged-ac2096e1-702f-4db7-8fe1-f30db9f79f04 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:08:30.229 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-c682cc1c-d6a3-435d-8d87-05eb01e1f1f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:30.229 4076 3051114314568 MainThread DEBUG oslo_concurrency.lockutils [req-c682cc1c-d6a3-435d-8d87-05eb01e1f1f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:30.229 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-c682cc1c-d6a3-435d-8d87-05eb01e1f1f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Processing event network-vif-plugged-ac2096e1-702f-4db7-8fe1-f30db9f79f04 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:08:30.280 4076 3051114313864 MainThread INFO nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Using config drive for instance 2021-09-10 17:08:30.986 4076 3051114313864 MainThread INFO nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Creating config drive at C:\openstack\instances\instance-0000006e\configdrive.iso 2021-09-10 17:08:31.010 4076 3051114313864 MainThread DEBUG oslo_concurrency.processutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpv5uhcwql execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:08:31.338 4076 3051114313864 MainThread DEBUG oslo_concurrency.processutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpv5uhcwql" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:08:31.778 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Setting boot order for instance: instance-0000006e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:08:31.828 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:08:31.872 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': '4268E32A-160E-4D93-905A-4DABB845D54D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19571), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:32.006 4076 3051114314568 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:32.006 4076 3051114314568 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:32.011 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:32.011 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:32.011 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4268E32A-160E-4D93-905A-4DABB845D54D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115794), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:32.011 4076 3051114313864 MainThread DEBUG os_win._utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1831343173980713 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:32.061 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-e5505fd8-e828-49dd-a7c9-1efe0188eabf - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:32.061 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-e5505fd8-e828-49dd-a7c9-1efe0188eabf - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:32.115 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-e5505fd8-e828-49dd-a7c9-1efe0188eabf - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:32.145 4076 3051114314568 MainThread INFO nova.compute.manager [req-e5505fd8-e828-49dd-a7c9-1efe0188eabf - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:32.311 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Successfully updated port: 2dbc6c2e-1121-4016-ab40-496deb947b2a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:08:32.311 4076 3051093462184 GreenThread-99 DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Acquired lock "refresh_cache-92438189-b8db-4465-b432-47722bf0d84c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:08:32.311 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:08:32.680 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:08:32.701 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Received event network-changed-2dbc6c2e-1121-4016-ab40-496deb947b2a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:08:32.701 4076 3051114314920 MainThread DEBUG nova.compute.manager [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Refreshing instance network info cache due to event network-changed-2dbc6c2e-1121-4016-ab40-496deb947b2a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:08:33.041 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': '92A0D015-D38C-479B-B423-F2EF7D5B5EFD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14888), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:33.183 4076 3051114314568 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:33.183 4076 3051114314568 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:33.183 4076 3051113432872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:33.183 4076 3051113432872 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:33.211 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '92A0D015-D38C-479B-B423-F2EF7D5B5EFD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114587), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:33.211 4076 3051114313864 MainThread DEBUG os_win._utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.383014440536499 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:33.223 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-1cc461b8-a88c-4bc5-afad-ebbc9789e703 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:33.223 4076 3051114314568 MainThread DEBUG nova.virt.hyperv.vmops [req-1cc461b8-a88c-4bc5-afad-ebbc9789e703 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:33.276 4076 3051114314568 MainThread DEBUG nova.compute.manager [req-1cc461b8-a88c-4bc5-afad-ebbc9789e703 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:33.305 4076 3051114314568 MainThread INFO nova.compute.manager [req-1cc461b8-a88c-4bc5-afad-ebbc9789e703 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:33.428 4076 3051093462184 GreenThread-99 DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Updating instance_info_cache with network_info: [{"id": "2dbc6c2e-1121-4016-ab40-496deb947b2a", "address": "fa:16:3e:cf:66:09", "network": {"id": "d6df0583-126e-43a0-b521-6e515b4cb0f8", "bridge": null, "label": "tempest-network-smoke--1432540075", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2dbc6c2e-11", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:33.459 4076 3051093462184 GreenThread-99 DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Releasing lock "refresh_cache-92438189-b8db-4465-b432-47722bf0d84c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:08:33.459 4076 3051093462184 GreenThread-99 DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Instance network_info: |[{"id": "2dbc6c2e-1121-4016-ab40-496deb947b2a", "address": "fa:16:3e:cf:66:09", "network": {"id": "d6df0583-126e-43a0-b521-6e515b4cb0f8", "bridge": null, "label": "tempest-network-smoke--1432540075", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2dbc6c2e-11", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:08:33.459 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-92438189-b8db-4465-b432-47722bf0d84c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:08:33.459 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Refreshing network info cache for port 2dbc6c2e-1121-4016-ab40-496deb947b2a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:08:33.459 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Preparing to wait for external event network-vif-plugged-2dbc6c2e-1121-4016-ab40-496deb947b2a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:08:33.466 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:33.466 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:33.466 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Creating VM instance-0000006f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:08:34.293 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': 'B94F7737-067D-4BB2-82E6-6D11F71DF39E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16337), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:34.449 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B94F7737-067D-4BB2-82E6-6D11F71DF39E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=118602), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:34.449 4076 3051114313864 MainThread DEBUG os_win._utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6216840744018555 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:34.449 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:34.449 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:34.449 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:34.449 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:34.467 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-7b4cfeb9-59cc-4b7f-93c1-70d86ef4efc2 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:34.467 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-7b4cfeb9-59cc-4b7f-93c1-70d86ef4efc2 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:34.497 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:08:34.526 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-7b4cfeb9-59cc-4b7f-93c1-70d86ef4efc2 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:34.570 4076 3051113163592 MainThread INFO nova.compute.manager [req-7b4cfeb9-59cc-4b7f-93c1-70d86ef4efc2 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:35.473 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': '11130BA5-919F-4EEF-A58B-56BEC0102593', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16042), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:35.541 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Updated VIF entry in instance network info cache for port 2dbc6c2e-1121-4016-ab40-496deb947b2a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:08:35.561 4076 3051114314920 MainThread DEBUG nova.network.neutron [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Updating instance_info_cache with network_info: [{"id": "2dbc6c2e-1121-4016-ab40-496deb947b2a", "address": "fa:16:3e:cf:66:09", "network": {"id": "d6df0583-126e-43a0-b521-6e515b4cb0f8", "bridge": null, "label": "tempest-network-smoke--1432540075", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2dbc6c2e-11", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:35.561 4076 3051114314920 MainThread DEBUG oslo_concurrency.lockutils [req-04531b47-58f6-42cf-b0bd-fca63c231561 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-92438189-b8db-4465-b432-47722bf0d84c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:08:35.624 4076 3051113162888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:35.624 4076 3051113162888 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:35.624 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:35.624 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:35.624 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '11130BA5-919F-4EEF-A58B-56BEC0102593', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113558), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:35.624 4076 3051114313864 MainThread DEBUG os_win._utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7965691089630127 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:35.639 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-511ae4a7-3ee9-4355-8290-66fbaedec5dc - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:35.639 4076 3051113162888 MainThread DEBUG nova.virt.hyperv.vmops [req-511ae4a7-3ee9-4355-8290-66fbaedec5dc - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:35.700 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-511ae4a7-3ee9-4355-8290-66fbaedec5dc - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:35.736 4076 3051113162888 MainThread INFO nova.compute.manager [req-511ae4a7-3ee9-4355-8290-66fbaedec5dc - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:35.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:36.654 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': 'ABF60426-799C-4DB6-83FE-6D67AF05EADC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17155), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:36.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:36.774 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:08:36.794 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'ABF60426-799C-4DB6-83FE-6D67AF05EADC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=122524), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:36.794 4076 3051114313864 MainThread DEBUG os_win._utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.966586351394653 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:36.822 4076 3051113162888 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:36.822 4076 3051113162888 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:36.822 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:36.822 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:36.856 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-389c9eb0-0902-40ff-8976-e562e0112574 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:36.856 4076 3051113162888 MainThread DEBUG nova.virt.hyperv.vmops [req-389c9eb0-0902-40ff-8976-e562e0112574 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:36.900 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-389c9eb0-0902-40ff-8976-e562e0112574 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:36.961 4076 3051113162888 MainThread INFO nova.compute.manager [req-389c9eb0-0902-40ff-8976-e562e0112574 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:37.244 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:08:37.244 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:08:37.821 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': '6683F0F5-463B-40D8-A48E-815BE9F3278A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16361), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:37.952 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:37.952 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:37.952 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:37.977 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] VM Stopped (Lifecycle Event) 2021-09-10 17:08:37.985 4076 3051114313864 MainThread ERROR os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006e\' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)instance-0000006eD4EBF031-5A6B-4773-96C6-5E261AA5866A%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6683F0F5-463B-40D8-A48E-815BE9F3278A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116911), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\nNot enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)\n\n'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A)", 'ErrorSummaryDescription': "'instance-0000006e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:37.985 4076 3051114313864 MainThread ERROR nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Failed to change vm state of instance-0000006e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. 2021-09-10 17:08:37.985 4076 3051114313864 MainThread INFO nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Got request to destroy instance 2021-09-10 17:08:37.997 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-b0b3d3aa-145f-423f-8ce4-9e2f5b02e777 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:37.997 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-b0b3d3aa-145f-423f-8ce4-9e2f5b02e777 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:38.027 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-b0b3d3aa-145f-423f-8ce4-9e2f5b02e777 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:38.092 4076 3051113163592 MainThread INFO nova.compute.manager [req-b0b3d3aa-145f-423f-8ce4-9e2f5b02e777 - - - - -] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:38.848 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:38.869 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:08:38.869 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:08:39.837 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:08:39.837 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:39.837 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:39.855 4076 3051114313864 MainThread DEBUG os_win.utils.compute.vmutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Successfully changed vm state of instance-0000006e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:08:39.855 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Successfully changed state of VM instance-0000006e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:08:39.888 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Waiting for WMI job: {'InstanceID': '6D7349CD-8FA5-4EB0-9E95-E3DCF181397A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14981), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:40.010 4076 3051114313864 MainThread DEBUG os_win.utils.jobutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] WMI job succeeded: {'InstanceID': '6D7349CD-8FA5-4EB0-9E95-E3DCF181397A', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29469), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:08:40.010 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:40.010 4076 3051114313864 MainThread DEBUG os_win.utils.pathutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Removing directory: C:\openstack\instances\instance-0000006e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:08:40.010 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:40.010 4076 3051114313864 MainThread ERROR nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Traceback (most recent call last): 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(*args, **kwargs) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] admin_password, network_info, block_device_info) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(self, *args, **kwds) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] should_plug_vifs=False) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'req_state': req_state}) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] time.sleep(sleep_time) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return f(*args, **kwargs) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return self._wait_for_job(job_path) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] error_desc=err_desc) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During handling of the above exception, another exception occurred: 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Traceback (most recent call last): 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] yield resources 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] accel_info=accel_info) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(*args, **kwargs) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] admin_password, network_info, block_device_info) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(self, *args, **kwds) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] should_plug_vifs=False) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'req_state': req_state}) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] time.sleep(sleep_time) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return f(*args, **kwargs) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return self._wait_for_job(job_path) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] error_desc=err_desc) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.010 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.010 4076 3051114313864 MainThread INFO nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Terminating instance 2021-09-10 17:08:40.010 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:08:40.010 4076 3051114313864 MainThread INFO nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Got request to destroy instance 2021-09-10 17:08:40.036 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.vmops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:08:40.036 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:40.036 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "instance-0000006e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:40.036 4076 3051114313864 MainThread INFO nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:08:40.036 4076 3051114313864 MainThread DEBUG nova.compute.claims [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:08:40.036 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:40.200 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.hostops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:40.234 4076 3051114313864 MainThread DEBUG nova.virt.hyperv.hostops [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:40.438 4076 3051114313864 MainThread DEBUG nova.compute.provider_tree [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:08:40.438 4076 3051114313864 MainThread DEBUG nova.scheduler.client.report [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:08:40.438 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:40.438 4076 3051114313864 MainThread ERROR nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Traceback (most recent call last): 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(*args, **kwargs) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] admin_password, network_info, block_device_info) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(self, *args, **kwds) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] should_plug_vifs=False) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'req_state': req_state}) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] time.sleep(sleep_time) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return f(*args, **kwargs) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return self._wait_for_job(job_path) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] error_desc=err_desc) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] During handling of the above exception, another exception occurred: 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Traceback (most recent call last): 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] accel_info=accel_info) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(*args, **kwargs) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] admin_password, network_info, block_device_info) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return function(self, *args, **kwds) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] should_plug_vifs=False) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'req_state': req_state}) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] time.sleep(sleep_time) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self.force_reraise() 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] raise self.value 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return f(*args, **kwargs) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] return self._wait_for_job(job_path) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] error_desc=err_desc) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. 2021-09-10 17:08:40.438 4076 3051114313864 MainThread TRACE nova.compute.manager [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] 2021-09-10 17:08:40.460 4076 3051114313864 MainThread DEBUG nova.compute.utils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:08:40.460 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Build of instance 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006e' failed to start. Not enough memory in the system to start the virtual machine instance-0000006e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006e' failed to start. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Not enough memory in the system to start the virtual machine instance-0000006e with ram size 1024 megabytes. (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) 'instance-0000006e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D4EBF031-5A6B-4773-96C6-5E261AA5866A) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:08:40.465 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:08:40.465 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:08:40.465 4076 3051114313864 MainThread DEBUG nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:08:40.465 4076 3051114313864 MainThread DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:08:40.648 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-c281bdea-ec28-4029-bf1b-c1f82781fb6b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Received event network-vif-plugged-2dbc6c2e-1121-4016-ab40-496deb947b2a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:08:40.648 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-c281bdea-ec28-4029-bf1b-c1f82781fb6b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:40.648 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-c281bdea-ec28-4029-bf1b-c1f82781fb6b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:40.648 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-c281bdea-ec28-4029-bf1b-c1f82781fb6b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Processing event network-vif-plugged-2dbc6c2e-1121-4016-ab40-496deb947b2a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:08:40.705 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Using config drive for instance 2021-09-10 17:08:41.200 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Creating config drive at C:\openstack\instances\instance-0000006f\configdrive.iso 2021-09-10 17:08:41.214 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3re9dbke execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:08:41.539 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000006f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp3re9dbke" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:08:41.929 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Setting boot order for instance: instance-0000006f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:08:41.979 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:08:42.001 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': 'F6D47002-EF23-4D0B-B1F0-26442DEB670D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16264), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:42.146 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:42.146 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:42.161 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:42.161 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:42.182 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-61c39370-64c9-428e-8c20-ef3bdde6c995 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:42.182 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-61c39370-64c9-428e-8c20-ef3bdde6c995 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:42.225 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-61c39370-64c9-428e-8c20-ef3bdde6c995 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:42.258 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F6D47002-EF23-4D0B-B1F0-26442DEB670D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=124241), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:42.258 4076 3051113161128 MainThread DEBUG os_win._utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.2790532112121582 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:42.277 4076 3051113163768 MainThread INFO nova.compute.manager [req-61c39370-64c9-428e-8c20-ef3bdde6c995 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:42.978 4076 3051114313864 MainThread DEBUG nova.network.neutron [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:42.998 4076 3051114313864 MainThread INFO nova.compute.manager [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] [instance: 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a] Took 2.53 seconds to deallocate network for instance. 2021-09-10 17:08:43.233 4076 3051114313864 MainThread INFO nova.scheduler.client.report [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Deleted allocations for instance 6e1d1124-996e-4d4c-80bd-1aab17ba5d1a 2021-09-10 17:08:43.274 4076 3051114313864 MainThread DEBUG oslo_concurrency.lockutils [req-7e465c70-92e6-4951-a03b-c6338d90a206 c63d410ca6d947fcba0acf004916fa12 203bf8c8ff8149d18d5c8dee127f7611 - default default] Lock "6e1d1124-996e-4d4c-80bd-1aab17ba5d1a" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 29.046s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:43.285 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': 'CBE83976-3C28-4669-A304-87E6DC6AC8F6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15372), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:43.406 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:43.406 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:43.427 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:43.427 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:43.427 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CBE83976-3C28-4669-A304-87E6DC6AC8F6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111482), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:43.427 4076 3051113161128 MainThread DEBUG os_win._utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4479751586914062 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:43.476 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-40f863bb-d962-4603-8e7e-b01bd85b1e74 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:43.476 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-40f863bb-d962-4603-8e7e-b01bd85b1e74 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:43.517 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-40f863bb-d962-4603-8e7e-b01bd85b1e74 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:43.547 4076 3051113164648 MainThread INFO nova.compute.manager [req-40f863bb-d962-4603-8e7e-b01bd85b1e74 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:43.775 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:43.775 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:43.840 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:43.857 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:44.472 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '9D924FB0-7627-4A3D-83C2-0412CB54AFBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15179), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:44.604 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:44.604 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:44.604 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:44.604 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:44.604 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9D924FB0-7627-4A3D-83C2-0412CB54AFBC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116877), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:44.604 4076 3051113161128 MainThread DEBUG os_win._utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.6245687007904053 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:44.642 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-20802d71-e300-4f86-8ba1-17aea0b386ff - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:44.642 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-20802d71-e300-4f86-8ba1-17aea0b386ff - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:44.684 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-20802d71-e300-4f86-8ba1-17aea0b386ff - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:44.738 4076 3051113164648 MainThread INFO nova.compute.manager [req-20802d71-e300-4f86-8ba1-17aea0b386ff - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:44.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:44.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:08:45.634 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '6F3B96AB-3669-45FF-AB8E-0ED8FDCECFB4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14724), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:45.758 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:45.758 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:45.758 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:45.758 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:45.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:45.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:45.772 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6F3B96AB-3669-45FF-AB8E-0ED8FDCECFB4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111711), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:45.772 4076 3051113161128 MainThread DEBUG os_win._utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7926766872406006 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:45.791 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:45.791 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:45.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:08:45.791 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:45.791 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-d2211bfe-867b-4097-bd2a-fc8d266cf753 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:45.791 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-d2211bfe-867b-4097-bd2a-fc8d266cf753 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:45.815 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:45.840 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-d2211bfe-867b-4097-bd2a-fc8d266cf753 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:45.874 4076 3051113164648 MainThread INFO nova.compute.manager [req-d2211bfe-867b-4097-bd2a-fc8d266cf753 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:46.017 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2189MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:08:46.017 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.336 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 92438189-b8db-4465-b432-47722bf0d84c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:08:46.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:08:46.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:08:46.405 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:46.418 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:46.610 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:08:46.654 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:08:46.654 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:08:46.654 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.640s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:46.806 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '36379E36-78E8-4FB1-BA21-4DA608A50601', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14451), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:46.923 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:46.923 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:46.923 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:46.944 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:46.950 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '36379E36-78E8-4FB1-BA21-4DA608A50601', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112261), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:46.950 4076 3051113161128 MainThread DEBUG os_win._utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.970925331115723 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:08:46.963 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-b1a029a3-dbe6-408a-9fed-382296b7e5df - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:46.963 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-b1a029a3-dbe6-408a-9fed-382296b7e5df - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:47.008 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-b1a029a3-dbe6-408a-9fed-382296b7e5df - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:47.043 4076 3051113164648 MainThread INFO nova.compute.manager [req-b1a029a3-dbe6-408a-9fed-382296b7e5df - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:47.977 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '031CF655-B97F-4092-BDAA-6B99ADABEC80', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15524), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:48.105 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:08:48.105 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 92438189-b8db-4465-b432-47722bf0d84c] VM Stopped (Lifecycle Event) 2021-09-10 17:08:48.105 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:48.105 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:48.105 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-0000006f\' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA395590710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000006f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)instance-0000006fE72AC3EB-45C6-4902-A5B3-A62CA3955907%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '031CF655-B97F-4092-BDAA-6B99ADABEC80', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112042), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 8, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 8, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\nNot enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)\n\n'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907)", 'ErrorSummaryDescription': "'instance-0000006f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000006f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:08:48.105 4076 3051113161128 MainThread ERROR nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Failed to change vm state of instance-0000006f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. 2021-09-10 17:08:48.105 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Got request to destroy instance 2021-09-10 17:08:48.146 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a579c790-7da4-4f4c-a94d-6792750b8e91 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:08:48.146 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a579c790-7da4-4f4c-a94d-6792750b8e91 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:08:48.201 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a579c790-7da4-4f4c-a94d-6792750b8e91 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:08:48.228 4076 3051113164648 MainThread INFO nova.compute.manager [req-a579c790-7da4-4f4c-a94d-6792750b8e91 - - - - -] [instance: 92438189-b8db-4465-b432-47722bf0d84c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:08:48.666 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:08:49.930 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:08:49.930 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:49.930 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:49.951 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Successfully changed vm state of instance-0000006f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:08:49.951 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Successfully changed state of VM instance-0000006f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:08:50.006 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '3D7F8FF0-4C9F-47FE-9C71-2FF617AE3F3F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15544), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:08:50.132 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job succeeded: {'InstanceID': '3D7F8FF0-4C9F-47FE-9C71-2FF617AE3F3F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=32271), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:08:50.132 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:50.132 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Removing directory: C:\openstack\instances\instance-0000006f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:08:50.132 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:50.137 4076 3051113161128 MainThread ERROR nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Traceback (most recent call last): 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(*args, **kwargs) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] admin_password, network_info, block_device_info) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(self, *args, **kwds) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] should_plug_vifs=False) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'req_state': req_state}) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] time.sleep(sleep_time) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return f(*args, **kwargs) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return self._wait_for_job(job_path) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] error_desc=err_desc) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] During handling of the above exception, another exception occurred: 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Traceback (most recent call last): 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] yield resources 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] accel_info=accel_info) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(*args, **kwargs) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] admin_password, network_info, block_device_info) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(self, *args, **kwds) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] should_plug_vifs=False) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'req_state': req_state}) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] time.sleep(sleep_time) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return f(*args, **kwargs) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return self._wait_for_job(job_path) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] error_desc=err_desc) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.137 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.137 4076 3051113161128 MainThread INFO nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Terminating instance 2021-09-10 17:08:50.137 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:08:50.137 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Got request to destroy instance 2021-09-10 17:08:50.159 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:08:50.159 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:50.159 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-0000006f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:50.159 4076 3051113161128 MainThread INFO nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:08:50.159 4076 3051113161128 MainThread DEBUG nova.compute.claims [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:08:50.159 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:08:50.337 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:08:50.341 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:08:50.554 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:08:50.574 4076 3051113161128 MainThread ERROR nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Traceback (most recent call last): 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(*args, **kwargs) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] admin_password, network_info, block_device_info) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(self, *args, **kwds) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] should_plug_vifs=False) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'req_state': req_state}) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] time.sleep(sleep_time) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return f(*args, **kwargs) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return self._wait_for_job(job_path) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] error_desc=err_desc) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] During handling of the above exception, another exception occurred: 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Traceback (most recent call last): 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] accel_info=accel_info) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(*args, **kwargs) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] admin_password, network_info, block_device_info) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return function(self, *args, **kwds) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] should_plug_vifs=False) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'req_state': req_state}) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] time.sleep(sleep_time) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self.force_reraise() 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] raise self.value 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return f(*args, **kwargs) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] return self._wait_for_job(job_path) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] error_desc=err_desc) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. 2021-09-10 17:08:50.574 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 92438189-b8db-4465-b432-47722bf0d84c] 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Build of instance 92438189-b8db-4465-b432-47722bf0d84c was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000006f' failed to start. Not enough memory in the system to start the virtual machine instance-0000006f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000006f' failed to start. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Not enough memory in the system to start the virtual machine instance-0000006f with ram size 1024 megabytes. (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) 'instance-0000006f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E72AC3EB-45C6-4902-A5B3-A62CA3955907) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:08:50.574 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:08:52.707 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:08:52.752 4076 3051113161128 MainThread INFO nova.compute.manager [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 92438189-b8db-4465-b432-47722bf0d84c] Took 2.17 seconds to deallocate network for instance. 2021-09-10 17:08:52.996 4076 3051113161128 MainThread INFO nova.scheduler.client.report [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Deleted allocations for instance 92438189-b8db-4465-b432-47722bf0d84c 2021-09-10 17:08:53.039 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-de8cde5f-035f-44e3-8527-8057c740c302 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "92438189-b8db-4465-b432-47722bf0d84c" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:02.972 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:02.996 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:09:03.151 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:03.151 4076 3051113163768 MainThread DEBUG nova.virt.hardware [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:09:03.151 4076 3051113163768 MainThread INFO nova.compute.claims [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Claim successful on node n-h1-808474-1 2021-09-10 17:09:03.353 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:09:03.384 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:09:03.605 4076 3051113163768 MainThread DEBUG nova.compute.provider_tree [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:09:03.637 4076 3051113163768 MainThread DEBUG nova.scheduler.client.report [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:09:03.637 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:03.637 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:09:03.788 4076 3051113163768 MainThread DEBUG nova.compute.utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:09:03.788 4076 3051113164648 GreenThread-100 DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:09:03.788 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:09:03.808 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:09:04.087 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:09:04.103 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Spawning new instance 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG nova.block_device [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG os_win.utils.pathutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Creating directory: C:\openstack\instances\instance-00000070 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:09:04.107 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000070\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:09:04.138 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Resizing VHD C:\openstack\instances\instance-00000070\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:09:04.868 4076 3051113164648 GreenThread-100 DEBUG nova.policy [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '2d3b9159333f49ce9b3eb8da13bb8c09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cacc6af2801d4a19904b570d39d5bf31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:09:07.460 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Successfully created port: b091e489-e84b-44cf-a1ee-f57a6b4b1732 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:09:10.104 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Successfully updated port: b091e489-e84b-44cf-a1ee-f57a6b4b1732 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:09:10.121 4076 3051113164648 GreenThread-100 DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Acquired lock "refresh_cache-6932cca3-947d-4279-bafa-391b541d7d9c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:09:10.121 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:09:10.489 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:09:10.833 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Received event network-changed-b091e489-e84b-44cf-a1ee-f57a6b4b1732 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:09:10.833 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Refreshing instance network info cache due to event network-changed-b091e489-e84b-44cf-a1ee-f57a6b4b1732. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:09:12.054 4076 3051113164648 GreenThread-100 DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Updating instance_info_cache with network_info: [{"id": "b091e489-e84b-44cf-a1ee-f57a6b4b1732", "address": "fa:16:3e:29:91:e4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.254", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb091e489-e8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:09:12.100 4076 3051113164648 GreenThread-100 DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Releasing lock "refresh_cache-6932cca3-947d-4279-bafa-391b541d7d9c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:09:12.100 4076 3051113164648 GreenThread-100 DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Instance network_info: |[{"id": "b091e489-e84b-44cf-a1ee-f57a6b4b1732", "address": "fa:16:3e:29:91:e4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.254", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb091e489-e8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:09:12.100 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6932cca3-947d-4279-bafa-391b541d7d9c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:09:12.100 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Refreshing network info cache for port b091e489-e84b-44cf-a1ee-f57a6b4b1732 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:09:12.100 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Preparing to wait for external event network-vif-plugged-b091e489-e84b-44cf-a1ee-f57a6b4b1732 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:09:12.100 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:12.100 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:12.100 4076 3051113163768 MainThread DEBUG os_win.utils.compute.vmutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Creating VM instance-00000070 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:09:12.985 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:09:13.664 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Updated VIF entry in instance network info cache for port b091e489-e84b-44cf-a1ee-f57a6b4b1732. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:09:13.664 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Updating instance_info_cache with network_info: [{"id": "b091e489-e84b-44cf-a1ee-f57a6b4b1732", "address": "fa:16:3e:29:91:e4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.254", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb091e489-e8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:09:13.681 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-25fa20c1-701d-4fa3-9ecb-16a48e63908f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6932cca3-947d-4279-bafa-391b541d7d9c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:09:18.622 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-1fcced16-a255-4dca-adbd-711a3e846f42 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Received event network-vif-plugged-b091e489-e84b-44cf-a1ee-f57a6b4b1732 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:09:18.622 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-1fcced16-a255-4dca-adbd-711a3e846f42 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:18.622 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-1fcced16-a255-4dca-adbd-711a3e846f42 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:18.622 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-1fcced16-a255-4dca-adbd-711a3e846f42 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Processing event network-vif-plugged-b091e489-e84b-44cf-a1ee-f57a6b4b1732 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:09:18.658 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Using config drive for instance 2021-09-10 17:09:19.281 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Creating config drive at C:\openstack\instances\instance-00000070\configdrive.iso 2021-09-10 17:09:19.310 4076 3051113163768 MainThread DEBUG oslo_concurrency.processutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000070\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmpc147_g execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:09:19.634 4076 3051113163768 MainThread DEBUG oslo_concurrency.processutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000070\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmpc147_g" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:09:20.009 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Setting boot order for instance: instance-00000070: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:09:20.071 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:09:20.102 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': '24F4E800-2EA0-491C-9917-B5DFCCF26AFE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14735), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:20.219 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:20.219 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:20.242 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:20.242 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:20.246 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '24F4E800-2EA0-491C-9917-B5DFCCF26AFE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112250), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:20.246 4076 3051113163768 MainThread DEBUG os_win._utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1744394302368164 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:09:20.269 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ab528df8-ec73-4456-821b-579f4ffb466c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:20.269 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ab528df8-ec73-4456-821b-579f4ffb466c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:20.310 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ab528df8-ec73-4456-821b-579f4ffb466c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:20.329 4076 3051113164648 MainThread INFO nova.compute.manager [req-ab528df8-ec73-4456-821b-579f4ffb466c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:21.261 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': 'EB17E9C3-C16E-4015-B9DF-032941F20268', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16037), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:21.387 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:21.406 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:21.406 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:21.406 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:21.406 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EB17E9C3-C16E-4015-B9DF-032941F20268', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110813), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:21.406 4076 3051113163768 MainThread DEBUG os_win._utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3348214626312256 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:09:21.446 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-aa341a08-13fb-40fd-990c-a97f7104d801 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:21.446 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-aa341a08-13fb-40fd-990c-a97f7104d801 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:21.491 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-aa341a08-13fb-40fd-990c-a97f7104d801 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:21.538 4076 3051113164648 MainThread INFO nova.compute.manager [req-aa341a08-13fb-40fd-990c-a97f7104d801 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:22.449 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': 'F9A556D1-B21A-4D37-8176-8B1B6DE2C26A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14761), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:22.573 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:22.573 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:22.573 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:22.573 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:22.573 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F9A556D1-B21A-4D37-8176-8B1B6DE2C26A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110771), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:22.573 4076 3051113163768 MainThread DEBUG os_win._utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5014002323150635 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:09:22.618 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-be1da263-450b-465f-88d4-1aab43a89eb7 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:22.618 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-be1da263-450b-465f-88d4-1aab43a89eb7 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:22.663 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-be1da263-450b-465f-88d4-1aab43a89eb7 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:22.687 4076 3051113164648 MainThread INFO nova.compute.manager [req-be1da263-450b-465f-88d4-1aab43a89eb7 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:23.649 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': '6C7A45A6-58D6-41D5-9583-6CDACF0A17B5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15307), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:23.785 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:23.785 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:23.785 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:23.785 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:23.796 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6C7A45A6-58D6-41D5-9583-6CDACF0A17B5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109571), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:23.796 4076 3051113163768 MainThread DEBUG os_win._utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7243645191192627 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:09:23.817 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-00071af7-cd8e-41bd-a838-1d58a33f090a - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:23.817 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-00071af7-cd8e-41bd-a838-1d58a33f090a - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:23.839 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-00071af7-cd8e-41bd-a838-1d58a33f090a - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:23.878 4076 3051113164648 MainThread INFO nova.compute.manager [req-00071af7-cd8e-41bd-a838-1d58a33f090a - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:24.817 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': '4DC07BF4-1C1D-47FB-807B-E00E43CE5B11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14746), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:24.964 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:24.964 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:24.964 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:24.964 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:24.964 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4DC07BF4-1C1D-47FB-807B-E00E43CE5B11', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112694), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:24.964 4076 3051113163768 MainThread DEBUG os_win._utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.89279842376709 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:09:24.987 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-2f5f7357-e8aa-4062-952c-a06a1cefb6c5 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:24.987 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-2f5f7357-e8aa-4062-952c-a06a1cefb6c5 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:25.033 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-2f5f7357-e8aa-4062-952c-a06a1cefb6c5 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:25.044 4076 3051113164648 MainThread INFO nova.compute.manager [req-2f5f7357-e8aa-4062-952c-a06a1cefb6c5 - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:26.006 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': 'D6A92386-DA53-4CAA-990A-9941F2040ACE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14905), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:26.141 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:26.141 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:26.141 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:09:26.141 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] VM Stopped (Lifecycle Event) 2021-09-10 17:09:26.152 4076 3051113163768 MainThread ERROR os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000070\' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E2710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000070\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)instance-00000070DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D6A92386-DA53-4CAA-990A-9941F2040ACE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112836), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 9, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 9, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\nNot enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)\n\n'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27)", 'ErrorSummaryDescription': "'instance-00000070' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000070.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:09:26.152 4076 3051113163768 MainThread ERROR nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Failed to change vm state of instance-00000070 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. 2021-09-10 17:09:26.152 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Got request to destroy instance 2021-09-10 17:09:26.178 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-68fbdbcf-9ab7-448e-bc47-a6bf1a49dd6c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:09:26.178 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-68fbdbcf-9ab7-448e-bc47-a6bf1a49dd6c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:09:26.218 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-68fbdbcf-9ab7-448e-bc47-a6bf1a49dd6c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:09:26.263 4076 3051113164648 MainThread INFO nova.compute.manager [req-68fbdbcf-9ab7-448e-bc47-a6bf1a49dd6c - - - - -] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:09:27.965 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:09:27.965 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:27.965 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:27.991 4076 3051113163768 MainThread DEBUG os_win.utils.compute.vmutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Successfully changed vm state of instance-00000070 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:09:27.991 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Successfully changed state of VM instance-00000070 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:09:28.039 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Waiting for WMI job: {'InstanceID': 'D02AEABF-B2B9-4944-8AE1-5796E842523D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14386), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:09:28.179 4076 3051113163768 MainThread DEBUG os_win.utils.jobutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] WMI job succeeded: {'InstanceID': 'D02AEABF-B2B9-4944-8AE1-5796E842523D', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30212), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:09:28.179 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:28.179 4076 3051113163768 MainThread DEBUG os_win.utils.pathutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Removing directory: C:\openstack\instances\instance-00000070 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:09:28.179 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:28.179 4076 3051113163768 MainThread ERROR nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Traceback (most recent call last): 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(*args, **kwargs) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] admin_password, network_info, block_device_info) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(self, *args, **kwds) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] should_plug_vifs=False) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'req_state': req_state}) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] time.sleep(sleep_time) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return f(*args, **kwargs) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return self._wait_for_job(job_path) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] error_desc=err_desc) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During handling of the above exception, another exception occurred: 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Traceback (most recent call last): 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] yield resources 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] accel_info=accel_info) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(*args, **kwargs) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] admin_password, network_info, block_device_info) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(self, *args, **kwds) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] should_plug_vifs=False) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'req_state': req_state}) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] time.sleep(sleep_time) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return f(*args, **kwargs) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return self._wait_for_job(job_path) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] error_desc=err_desc) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.179 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.179 4076 3051113163768 MainThread INFO nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Terminating instance 2021-09-10 17:09:28.179 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:09:28.179 4076 3051113163768 MainThread INFO nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Got request to destroy instance 2021-09-10 17:09:28.198 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:09:28.198 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:28.198 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "instance-00000070" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:28.198 4076 3051113163768 MainThread INFO nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:09:28.198 4076 3051113163768 MainThread DEBUG nova.compute.claims [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:09:28.198 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:28.322 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:09:28.361 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.hostops [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:09:28.553 4076 3051113163768 MainThread DEBUG nova.compute.provider_tree [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.scheduler.client.report [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:28.574 4076 3051113163768 MainThread ERROR nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Traceback (most recent call last): 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(*args, **kwargs) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] admin_password, network_info, block_device_info) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(self, *args, **kwds) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] should_plug_vifs=False) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'req_state': req_state}) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] time.sleep(sleep_time) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return f(*args, **kwargs) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return self._wait_for_job(job_path) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] error_desc=err_desc) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] During handling of the above exception, another exception occurred: 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Traceback (most recent call last): 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] accel_info=accel_info) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(*args, **kwargs) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] admin_password, network_info, block_device_info) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return function(self, *args, **kwds) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] should_plug_vifs=False) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'req_state': req_state}) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] time.sleep(sleep_time) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self.force_reraise() 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] raise self.value 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return f(*args, **kwargs) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] return self._wait_for_job(job_path) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] error_desc=err_desc) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. 2021-09-10 17:09:28.574 4076 3051113163768 MainThread TRACE nova.compute.manager [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.compute.utils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Build of instance 6932cca3-947d-4279-bafa-391b541d7d9c was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000070' failed to start. Not enough memory in the system to start the virtual machine instance-00000070. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000070' failed to start. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Not enough memory in the system to start the virtual machine instance-00000070 with ram size 1024 megabytes. (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) 'instance-00000070' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DFE1817C-1D20-4CE1-89C0-3BFE4CFB5E27) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:09:28.574 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:09:31.241 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:09:31.263 4076 3051113163768 MainThread INFO nova.compute.manager [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] [instance: 6932cca3-947d-4279-bafa-391b541d7d9c] Took 2.69 seconds to deallocate network for instance. 2021-09-10 17:09:31.632 4076 3051113163768 MainThread INFO nova.scheduler.client.report [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Deleted allocations for instance 6932cca3-947d-4279-bafa-391b541d7d9c 2021-09-10 17:09:31.679 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-feaaa612-de03-49c4-9474-8b5fcd86216d 2d3b9159333f49ce9b3eb8da13bb8c09 cacc6af2801d4a19904b570d39d5bf31 - default default] Lock "6932cca3-947d-4279-bafa-391b541d7d9c" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 28.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:35.804 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:37.788 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:37.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:09:37.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 17:09:38.023 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:09:38.023 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:09:38.023 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 17:09:40.792 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:09:40.821 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:09:40.821 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:09:43.992 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:43.992 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:45.852 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:45.852 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:45.870 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:45.870 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:45.870 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:09:46.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:09:46.780 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:46.780 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:46.780 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:09:46.780 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:09:46.813 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:09:47.038 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2188MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:09:47.038 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:09:47.226 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:09:47.285 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:09:47.285 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:09:47.484 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:09:47.504 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:09:47.504 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:09:47.504 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:09:48.504 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:03.700 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:03.732 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:10:03.867 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:03.883 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:10:03.883 4076 3051113164648 MainThread INFO nova.compute.claims [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Claim successful on node n-h1-808474-1 2021-09-10 17:10:04.056 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:10:04.087 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:10:04.323 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:10:04.351 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:10:04.351 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.485s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:04.351 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:10:04.494 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:10:04.494 4076 3051113162184 GreenThread-101 DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:10:04.494 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:10:04.540 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:10:04.779 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:10:04.779 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Spawning new instance 2021-09-10 17:10:04.808 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:04.808 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:04.808 4076 3051113164648 MainThread DEBUG nova.block_device [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:10:04.808 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:04.808 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:04.820 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Creating directory: C:\openstack\instances\instance-00000071 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:10:04.820 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000071\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:10:04.838 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Resizing VHD C:\openstack\instances\instance-00000071\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:10:05.666 4076 3051113162184 GreenThread-101 DEBUG nova.policy [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '95e49dab1ffa48cfb40576e8646baad3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f4ebc509414a41b9890ca144a70c816d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:10:08.699 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Successfully created port: 618311b4-5dad-44c8-8306-432356ca4c5f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:10:11.003 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Successfully updated port: 618311b4-5dad-44c8-8306-432356ca4c5f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:10:11.022 4076 3051113162184 GreenThread-101 DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Acquired lock "refresh_cache-9f6a1261-e291-45ef-a4fd-539e3f9f4044" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:10:11.034 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:10:11.086 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Received event network-changed-618311b4-5dad-44c8-8306-432356ca4c5f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:10:11.086 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Refreshing instance network info cache due to event network-changed-618311b4-5dad-44c8-8306-432356ca4c5f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:10:11.292 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:10:12.914 4076 3051113162184 GreenThread-101 DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Updating instance_info_cache with network_info: [{"id": "618311b4-5dad-44c8-8306-432356ca4c5f", "address": "fa:16:3e:17:c3:f6", "network": {"id": "eb29e349-1649-4448-aead-199cc5a35342", "bridge": null, "label": "tempest-network-smoke--1956885507", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap618311b4-5d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:10:12.944 4076 3051113162184 GreenThread-101 DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Releasing lock "refresh_cache-9f6a1261-e291-45ef-a4fd-539e3f9f4044" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:10:12.944 4076 3051113162184 GreenThread-101 DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Instance network_info: |[{"id": "618311b4-5dad-44c8-8306-432356ca4c5f", "address": "fa:16:3e:17:c3:f6", "network": {"id": "eb29e349-1649-4448-aead-199cc5a35342", "bridge": null, "label": "tempest-network-smoke--1956885507", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap618311b4-5d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:10:12.944 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-9f6a1261-e291-45ef-a4fd-539e3f9f4044" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:10:12.944 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Refreshing network info cache for port 618311b4-5dad-44c8-8306-432356ca4c5f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:10:12.944 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Preparing to wait for external event network-vif-plugged-618311b4-5dad-44c8-8306-432356ca4c5f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:10:12.944 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:12.944 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:12.944 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Creating VM instance-00000071 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:10:13.820 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:10:15.573 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Updated VIF entry in instance network info cache for port 618311b4-5dad-44c8-8306-432356ca4c5f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:10:15.573 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Updating instance_info_cache with network_info: [{"id": "618311b4-5dad-44c8-8306-432356ca4c5f", "address": "fa:16:3e:17:c3:f6", "network": {"id": "eb29e349-1649-4448-aead-199cc5a35342", "bridge": null, "label": "tempest-network-smoke--1956885507", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "f4ebc509414a41b9890ca144a70c816d", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap618311b4-5d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:10:15.601 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-2b7eea09-ad6d-4326-aaee-5d9307d3f022 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-9f6a1261-e291-45ef-a4fd-539e3f9f4044" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:10:21.527 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-2c68b2c9-a59f-42d0-9a23-16d6042978be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Received event network-vif-plugged-618311b4-5dad-44c8-8306-432356ca4c5f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:10:21.527 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-2c68b2c9-a59f-42d0-9a23-16d6042978be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:21.527 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-2c68b2c9-a59f-42d0-9a23-16d6042978be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:21.527 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-2c68b2c9-a59f-42d0-9a23-16d6042978be 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Processing event network-vif-plugged-618311b4-5dad-44c8-8306-432356ca4c5f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:10:21.570 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Using config drive for instance 2021-09-10 17:10:22.539 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Creating config drive at C:\openstack\instances\instance-00000071\configdrive.iso 2021-09-10 17:10:22.571 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000071\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpbazewus6 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:10:22.882 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000071\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpbazewus6" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:10:23.265 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Setting boot order for instance: instance-00000071: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:10:23.295 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:10:23.342 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '61FBC6DE-626B-4B18-A095-AD1B4EAC5920', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18625), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:23.476 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '61FBC6DE-626B-4B18-A095-AD1B4EAC5920', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117283), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:23.476 4076 3051113164648 MainThread DEBUG os_win._utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18148183822631836 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:10:23.476 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:23.476 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:23.476 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:23.476 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:23.519 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-6d16a7d0-5652-400f-b4f9-3ed48febf126 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:23.519 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-6d16a7d0-5652-400f-b4f9-3ed48febf126 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:23.568 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-6d16a7d0-5652-400f-b4f9-3ed48febf126 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:23.586 4076 3051113161128 MainThread INFO nova.compute.manager [req-6d16a7d0-5652-400f-b4f9-3ed48febf126 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:24.523 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': 'B4DB0D9F-7FC5-4940-B328-CFAFF5A15B38', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15068), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:24.633 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:24.662 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:24.662 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:24.662 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:24.678 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B4DB0D9F-7FC5-4940-B328-CFAFF5A15B38', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111003), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:24.680 4076 3051113164648 MainThread DEBUG os_win._utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3854382038116455 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:10:24.680 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a23a3426-b2c2-4dc6-bd06-7e1b646207f6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:24.680 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a23a3426-b2c2-4dc6-bd06-7e1b646207f6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:24.728 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a23a3426-b2c2-4dc6-bd06-7e1b646207f6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:24.789 4076 3051113161128 MainThread INFO nova.compute.manager [req-a23a3426-b2c2-4dc6-bd06-7e1b646207f6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:25.711 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': 'DCC59281-0653-4746-82DB-690D805AC12E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14363), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:25.837 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:25.837 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:25.837 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:25.837 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:25.837 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DCC59281-0653-4746-82DB-690D805AC12E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111482), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:25.837 4076 3051113164648 MainThread DEBUG os_win._utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.542630195617676 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:10:25.867 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-10370e41-7157-447d-9dbb-0169d298067d - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:25.867 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-10370e41-7157-447d-9dbb-0169d298067d - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:25.903 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-10370e41-7157-447d-9dbb-0169d298067d - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:25.960 4076 3051113161128 MainThread INFO nova.compute.manager [req-10370e41-7157-447d-9dbb-0169d298067d - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:26.851 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': 'E63676DB-802E-4306-A53F-E41C9FAE62CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14923), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:26.995 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:26.995 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:26.995 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:26.995 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:26.995 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E63676DB-802E-4306-A53F-E41C9FAE62CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112235), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:26.995 4076 3051113164648 MainThread DEBUG os_win._utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7004640102386475 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:10:27.038 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-1c5d2e28-041c-43e8-847b-fdb6761004c6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:27.038 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-1c5d2e28-041c-43e8-847b-fdb6761004c6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:27.086 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-1c5d2e28-041c-43e8-847b-fdb6761004c6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:27.133 4076 3051113161128 MainThread INFO nova.compute.manager [req-1c5d2e28-041c-43e8-847b-fdb6761004c6 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:28.025 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '2AAF967E-6E21-459D-B45A-8A22A65C3FCB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17065), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:28.132 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:28.132 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:28.132 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:28.132 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:28.166 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2AAF967E-6E21-459D-B45A-8A22A65C3FCB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112118), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:28.166 4076 3051113164648 MainThread DEBUG os_win._utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.871095657348633 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:10:28.166 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-cf791bcf-38cb-41c6-8d73-755016dcdd02 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:28.166 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-cf791bcf-38cb-41c6-8d73-755016dcdd02 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:28.211 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-cf791bcf-38cb-41c6-8d73-755016dcdd02 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:28.242 4076 3051113161128 MainThread INFO nova.compute.manager [req-cf791bcf-38cb-41c6-8d73-755016dcdd02 - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:29.195 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '05E16C68-C4AF-4CAA-AC1D-449EE3A3D3AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14874), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:29.321 4076 3051113161128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:10:29.321 4076 3051113161128 MainThread INFO nova.compute.manager [-] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] VM Stopped (Lifecycle Event) 2021-09-10 17:10:29.321 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:29.321 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:29.343 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job failed: {'RawErrors': (('124\'instance-00000071\' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E1810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000071\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)instance-000000715A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '05E16C68-C4AF-4CAA-AC1D-449EE3A3D3AB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112888), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 10, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 10, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\nNot enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)\n\n'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18)", 'ErrorSummaryDescription': "'instance-00000071' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000071.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:10:29.343 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Failed to change vm state of instance-00000071 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. 2021-09-10 17:10:29.343 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Got request to destroy instance 2021-09-10 17:10:29.343 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-914debcc-ac9d-4fd4-b1bf-9d7206079a2c - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:10:29.343 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-914debcc-ac9d-4fd4-b1bf-9d7206079a2c - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:10:29.389 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-914debcc-ac9d-4fd4-b1bf-9d7206079a2c - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:10:29.436 4076 3051113161128 MainThread INFO nova.compute.manager [req-914debcc-ac9d-4fd4-b1bf-9d7206079a2c - - - - -] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:10:31.133 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:10:31.133 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:31.152 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:31.152 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Successfully changed vm state of instance-00000071 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:10:31.152 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Successfully changed state of VM instance-00000071 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:10:31.216 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Waiting for WMI job: {'InstanceID': '216C5DFB-32E5-4326-988D-8B9CDE254CDD', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14743), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:10:31.336 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] WMI job succeeded: {'InstanceID': '216C5DFB-32E5-4326-988D-8B9CDE254CDD', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28893), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:10:31.336 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:31.336 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Removing directory: C:\openstack\instances\instance-00000071 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:10:31.336 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:31.336 4076 3051113164648 MainThread ERROR nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Traceback (most recent call last): 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(*args, **kwargs) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] admin_password, network_info, block_device_info) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(self, *args, **kwds) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] should_plug_vifs=False) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'req_state': req_state}) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] time.sleep(sleep_time) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return f(*args, **kwargs) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return self._wait_for_job(job_path) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] error_desc=err_desc) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During handling of the above exception, another exception occurred: 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Traceback (most recent call last): 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] yield resources 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] accel_info=accel_info) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(*args, **kwargs) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] admin_password, network_info, block_device_info) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(self, *args, **kwds) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] should_plug_vifs=False) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'req_state': req_state}) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] time.sleep(sleep_time) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return f(*args, **kwargs) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return self._wait_for_job(job_path) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] error_desc=err_desc) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.336 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.336 4076 3051113164648 MainThread INFO nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Terminating instance 2021-09-10 17:10:31.336 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:10:31.336 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Got request to destroy instance 2021-09-10 17:10:31.351 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:10:31.351 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:31.351 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "instance-00000071" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:31.351 4076 3051113164648 MainThread INFO nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:10:31.351 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:10:31.351 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:31.616 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:10:31.632 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:10:31.822 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:31.860 4076 3051113164648 MainThread ERROR nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Traceback (most recent call last): 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(*args, **kwargs) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] admin_password, network_info, block_device_info) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(self, *args, **kwds) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] should_plug_vifs=False) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'req_state': req_state}) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] time.sleep(sleep_time) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return f(*args, **kwargs) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return self._wait_for_job(job_path) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] error_desc=err_desc) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] During handling of the above exception, another exception occurred: 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Traceback (most recent call last): 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] accel_info=accel_info) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(*args, **kwargs) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] admin_password, network_info, block_device_info) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return function(self, *args, **kwds) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] should_plug_vifs=False) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'req_state': req_state}) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] time.sleep(sleep_time) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self.force_reraise() 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] raise self.value 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return f(*args, **kwargs) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] return self._wait_for_job(job_path) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] error_desc=err_desc) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. 2021-09-10 17:10:31.860 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Build of instance 9f6a1261-e291-45ef-a4fd-539e3f9f4044 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000071' failed to start. Not enough memory in the system to start the virtual machine instance-00000071. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000071' failed to start. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Not enough memory in the system to start the virtual machine instance-00000071 with ram size 1024 megabytes. (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) 'instance-00000071' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5A6E8C4A-3B9B-41B2-95E9-7E2912FB7E18) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:10:31.860 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:10:34.992 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:10:35.015 4076 3051113164648 MainThread INFO nova.compute.manager [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] [instance: 9f6a1261-e291-45ef-a4fd-539e3f9f4044] Took 3.16 seconds to deallocate network for instance. 2021-09-10 17:10:35.366 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Deleted allocations for instance 9f6a1261-e291-45ef-a4fd-539e3f9f4044 2021-09-10 17:10:35.423 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a1e16595-e3ef-4589-a335-8a077525d0ae 95e49dab1ffa48cfb40576e8646baad3 f4ebc509414a41b9890ca144a70c816d - default default] Lock "9f6a1261-e291-45ef-a4fd-539e3f9f4044" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 31.735s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:37.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:39.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:39.773 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:10:40.703 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:10:40.726 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:10:43.174 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:10:43.194 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:10:43.194 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:10:45.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:45.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:45.826 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:46.788 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:46.819 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:46.823 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:46.823 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:10:47.819 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:48.788 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:10:48.794 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:48.794 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:10:48.794 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:10:48.794 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:10:48.819 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:10:49.038 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2205MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:10:49.038 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:10:49.280 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:10:49.319 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:10:49.346 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:10:49.539 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:10:49.570 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:10:49.586 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:10:49.586 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:11:40.571 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:40.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:40.773 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:11:40.977 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:11:40.977 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:11:42.570 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:11:42.586 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:11:42.586 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:11:45.585 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:46.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:46.789 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:46.789 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:11:47.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:47.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:47.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:47.774 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:11:48.005 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:48.821 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:48.836 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:48.836 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:11:48.836 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:11:48.836 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:11:48.836 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:11:48.883 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:11:49.102 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2220MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:11:49.102 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:11:50.024 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:11:50.135 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:11:50.194 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:11:50.194 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:11:50.253 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:11:50.337 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:11:50.337 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:11:50.352 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:11:50.555 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:11:50.570 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:11:50.570 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:11:50.570 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 1.468s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:11:51.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:59.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:11:59.786 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:11:59.805 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:12:40.800 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:40.800 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:12:41.289 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:12:41.289 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:12:42.946 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:12:42.960 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:12:42.960 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:12:42.960 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:46.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:46.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:47.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:47.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:47.773 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:12:48.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:49.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:50.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:50.876 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:12:50.914 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:12:50.914 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:12:50.914 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:12:50.914 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:12:50.958 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:12:51.176 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2223MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:12:51.176 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:12:51.507 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.507 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.507 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:12:51.525 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:12:51.568 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:12:51.584 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:12:51.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:12:51.804 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:12:51.804 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:12:51.804 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:13:41.695 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:41.695 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:13:42.243 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:13:42.243 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:13:45.179 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:13:45.211 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:13:45.211 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:13:45.211 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:47.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:47.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:47.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:48.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:48.784 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:13:48.987 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:49.794 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:50.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:52.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:13:52.787 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:13:52.787 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:13:52.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:13:52.787 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:13:52.836 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:13:53.052 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2227MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:13:53.052 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:13:53.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:13:53.479 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:13:53.499 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:13:53.681 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:13:53.722 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:13:53.722 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:13:53.722 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.672s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:14:41.720 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:41.720 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:14:43.189 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:14:43.189 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:14:46.991 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:14:47.033 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:14:47.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:14:47.033 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:48.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:48.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:49.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:49.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:49.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:14:50.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:50.783 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:51.798 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:54.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:14:54.802 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:14:54.802 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:14:54.802 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:14:54.802 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:14:54.802 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:14:55.037 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2229MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:14:55.037 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:14:55.346 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:14:55.397 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:14:55.412 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:14:55.615 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:14:55.631 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:14:55.631 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:14:55.631 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:30.300 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:30.354 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:15:30.581 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:30.581 4076 3051113162536 MainThread DEBUG nova.virt.hardware [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:15:30.581 4076 3051113162536 MainThread INFO nova.compute.claims [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Claim successful on node n-h1-808474-1 2021-09-10 17:15:30.735 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.hostops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:15:30.757 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.hostops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:15:30.954 4076 3051113162536 MainThread DEBUG nova.compute.provider_tree [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:15:30.969 4076 3051113162536 MainThread DEBUG nova.scheduler.client.report [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:15:30.969 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:30.969 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:15:31.175 4076 3051113162536 MainThread DEBUG nova.compute.utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:15:31.175 4076 3051113161128 GreenThread-102 DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:15:31.175 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:15:31.224 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:15:31.600 4076 3051113161128 GreenThread-102 DEBUG nova.policy [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '799e4a49be0e448b9d6d54d57b7d8d1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cf483b0ac5349bca4c898bb3b806d21', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:15:31.621 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:15:31.641 4076 3051113162536 MainThread INFO nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Spawning new instance 2021-09-10 17:15:31.647 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:31.647 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:31.647 4076 3051113162536 MainThread DEBUG nova.block_device [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:15:31.647 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:31.647 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:31.661 4076 3051113162536 MainThread DEBUG os_win.utils.pathutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating directory: C:\openstack\instances\instance-00000072 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:15:31.661 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000072\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:15:31.672 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Resizing VHD C:\openstack\instances\instance-00000072\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:15:32.882 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Successfully created port: d70324d5-8555-446b-ae0d-28f57cc10240 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:15:34.908 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Successfully updated port: d70324d5-8555-446b-ae0d-28f57cc10240 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:15:34.908 4076 3051113161128 GreenThread-102 DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Acquired lock "refresh_cache-236a5f41-195e-4bd9-9c70-230e5a1cf40f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:15:34.908 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:15:35.094 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:15:35.453 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Received event network-changed-d70324d5-8555-446b-ae0d-28f57cc10240 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:15:35.453 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Refreshing instance network info cache due to event network-changed-d70324d5-8555-446b-ae0d-28f57cc10240. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:15:36.026 4076 3051113161128 GreenThread-102 DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Updating instance_info_cache with network_info: [{"id": "d70324d5-8555-446b-ae0d-28f57cc10240", "address": "fa:16:3e:8d:e1:e7", "network": {"id": "a847bbb1-95d3-4aab-8de0-ad5d3af60dc3", "bridge": null, "label": "tempest-network-smoke--1228547857", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd70324d5-85", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:15:36.054 4076 3051113161128 GreenThread-102 DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Releasing lock "refresh_cache-236a5f41-195e-4bd9-9c70-230e5a1cf40f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:15:36.054 4076 3051113161128 GreenThread-102 DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Instance network_info: |[{"id": "d70324d5-8555-446b-ae0d-28f57cc10240", "address": "fa:16:3e:8d:e1:e7", "network": {"id": "a847bbb1-95d3-4aab-8de0-ad5d3af60dc3", "bridge": null, "label": "tempest-network-smoke--1228547857", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd70324d5-85", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:15:36.054 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Preparing to wait for external event network-vif-plugged-d70324d5-8555-446b-ae0d-28f57cc10240 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:15:36.054 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:36.054 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:36.054 4076 3051113162536 MainThread DEBUG os_win.utils.compute.vmutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating VM instance-00000072 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:15:36.054 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-236a5f41-195e-4bd9-9c70-230e5a1cf40f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:15:36.054 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Refreshing network info cache for port d70324d5-8555-446b-ae0d-28f57cc10240 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:15:36.956 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:15:37.209 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Updated VIF entry in instance network info cache for port d70324d5-8555-446b-ae0d-28f57cc10240. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:15:37.209 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Updating instance_info_cache with network_info: [{"id": "d70324d5-8555-446b-ae0d-28f57cc10240", "address": "fa:16:3e:8d:e1:e7", "network": {"id": "a847bbb1-95d3-4aab-8de0-ad5d3af60dc3", "bridge": null, "label": "tempest-network-smoke--1228547857", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd70324d5-85", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:15:37.225 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-2bdcd1fc-fd8c-4551-997f-679a981613f2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-236a5f41-195e-4bd9-9c70-230e5a1cf40f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:15:41.617 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:41.617 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:15:41.704 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:15:41.704 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:15:43.219 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:15:43.241 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:15:43.241 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:15:43.241 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:43.892 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-3cbd8558-7b5d-4b3c-83b8-9ef3e1fe9db7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Received event network-vif-plugged-d70324d5-8555-446b-ae0d-28f57cc10240 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:15:43.892 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-3cbd8558-7b5d-4b3c-83b8-9ef3e1fe9db7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:43.892 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-3cbd8558-7b5d-4b3c-83b8-9ef3e1fe9db7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:43.892 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-3cbd8558-7b5d-4b3c-83b8-9ef3e1fe9db7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Processing event network-vif-plugged-d70324d5-8555-446b-ae0d-28f57cc10240 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:15:43.938 4076 3051113162536 MainThread INFO nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Using config drive for instance 2021-09-10 17:15:44.248 4076 3051113162536 MainThread INFO nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Creating config drive at C:\openstack\instances\instance-00000072\configdrive.iso 2021-09-10 17:15:44.272 4076 3051113162536 MainThread DEBUG oslo_concurrency.processutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000072\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpn9v784dq execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:15:44.585 4076 3051113162536 MainThread DEBUG oslo_concurrency.processutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000072\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpn9v784dq" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:15:44.960 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Setting boot order for instance: instance-00000072: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:15:45.023 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:15:45.054 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '09DFC173-1489-4FA7-A9F7-2EB693436702', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18732), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:45.191 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:45.191 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:45.191 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:45.191 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:45.204 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '09DFC173-1489-4FA7-A9F7-2EB693436702', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113848), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:45.220 4076 3051113162536 MainThread DEBUG os_win._utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19706106185913086 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:15:45.220 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-041a3b8f-7a0d-41fc-8de9-182c19084d69 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:45.220 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-041a3b8f-7a0d-41fc-8de9-182c19084d69 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:45.272 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-041a3b8f-7a0d-41fc-8de9-182c19084d69 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:45.336 4076 3051113161304 MainThread INFO nova.compute.manager [req-041a3b8f-7a0d-41fc-8de9-182c19084d69 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:46.257 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '2B32BAC5-30E9-4101-BDC4-B75CFE4BB5C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19549), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:46.382 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:46.382 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:46.382 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:46.382 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:46.382 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2B32BAC5-30E9-4101-BDC4-B75CFE4BB5C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113962), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:46.382 4076 3051113162536 MainThread DEBUG os_win._utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3584976196289062 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:15:46.438 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-b6b823cb-060a-40d2-880f-3c7a6878e67d - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:46.438 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-b6b823cb-060a-40d2-880f-3c7a6878e67d - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:46.484 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-b6b823cb-060a-40d2-880f-3c7a6878e67d - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:46.521 4076 3051113161304 MainThread INFO nova.compute.manager [req-b6b823cb-060a-40d2-880f-3c7a6878e67d - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:47.453 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '5A732325-860D-45E3-8A1F-1048C92A9B68', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14804), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:47.572 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:47.572 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:47.572 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:47.572 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:47.601 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5A732325-860D-45E3-8A1F-1048C92A9B68', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112269), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:47.601 4076 3051113162536 MainThread DEBUG os_win._utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5776374340057373 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:15:47.641 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-8b1869a0-7747-4ca6-8362-d800338a3b16 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:47.641 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-8b1869a0-7747-4ca6-8362-d800338a3b16 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:47.678 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-8b1869a0-7747-4ca6-8362-d800338a3b16 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:47.716 4076 3051113161304 MainThread INFO nova.compute.manager [req-8b1869a0-7747-4ca6-8362-d800338a3b16 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:48.632 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '9A2E0600-605F-4894-A4F3-D33A610CC347', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14823), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:48.773 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:48.773 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:48.773 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:48.773 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:48.773 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9A2E0600-605F-4894-A4F3-D33A610CC347', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111413), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:48.773 4076 3051113162536 MainThread DEBUG os_win._utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7501533031463623 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:15:48.791 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-2190a641-a889-482c-94a0-f4a81087ae12 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:48.791 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-2190a641-a889-482c-94a0-f4a81087ae12 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:48.844 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-2190a641-a889-482c-94a0-f4a81087ae12 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:48.882 4076 3051113161304 MainThread INFO nova.compute.manager [req-2190a641-a889-482c-94a0-f4a81087ae12 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:48.984 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:48.984 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:49.795 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:49.795 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:15:49.803 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '5FE5F3F7-6C0A-48B7-B965-6BA80D921015', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16472), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:49.944 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:49.944 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:49.944 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:49.944 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:49.944 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5FE5F3F7-6C0A-48B7-B965-6BA80D921015', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111985), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:49.944 4076 3051113162536 MainThread DEBUG os_win._utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9212727546691895 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:15:49.960 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-b1e694a6-8d44-4818-9446-7cbbf3f57cb3 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:49.960 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-b1e694a6-8d44-4818-9446-7cbbf3f57cb3 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:50.003 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-b1e694a6-8d44-4818-9446-7cbbf3f57cb3 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:50.093 4076 3051113161304 MainThread INFO nova.compute.manager [req-b1e694a6-8d44-4818-9446-7cbbf3f57cb3 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:50.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:50.984 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '4A8DE0A4-2511-4B7C-A804-CF232EE8B890', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14671), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:51.115 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:15:51.115 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] VM Stopped (Lifecycle Event) 2021-09-10 17:15:51.115 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:51.115 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:51.115 4076 3051113162536 MainThread ERROR os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000072\' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000072\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)instance-00000072A6279449-6D43-447B-922B-A1A3259238CB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4A8DE0A4-2511-4B7C-A804-CF232EE8B890', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112343), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 15, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 15, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\nNot enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)\n\n'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB)", 'ErrorSummaryDescription': "'instance-00000072' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000072.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:15:51.115 4076 3051113162536 MainThread ERROR nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Failed to change vm state of instance-00000072 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. 2021-09-10 17:15:51.115 4076 3051113162536 MainThread INFO nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Got request to destroy instance 2021-09-10 17:15:51.141 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-29bbff10-26d6-4aad-b72c-0d48e31227e8 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:15:51.141 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-29bbff10-26d6-4aad-b72c-0d48e31227e8 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:15:51.178 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-29bbff10-26d6-4aad-b72c-0d48e31227e8 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:15:51.207 4076 3051113161304 MainThread INFO nova.compute.manager [req-29bbff10-26d6-4aad-b72c-0d48e31227e8 - - - - -] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:15:51.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:51.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:52.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:52.953 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:15:52.953 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:52.953 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:52.960 4076 3051113162536 MainThread DEBUG os_win.utils.compute.vmutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed vm state of instance-00000072 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:15:52.960 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed state of VM instance-00000072 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:15:53.012 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'E7FA48A5-7165-495D-B593-4CE4507344BA', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15181), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:15:53.141 4076 3051113162536 MainThread DEBUG os_win.utils.jobutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job succeeded: {'InstanceID': 'E7FA48A5-7165-495D-B593-4CE4507344BA', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29018), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:15:53.141 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:53.141 4076 3051113162536 MainThread DEBUG os_win.utils.pathutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Removing directory: C:\openstack\instances\instance-00000072 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:15:53.141 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:53.141 4076 3051113162536 MainThread ERROR nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Traceback (most recent call last): 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(*args, **kwargs) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] admin_password, network_info, block_device_info) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(self, *args, **kwds) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] should_plug_vifs=False) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'req_state': req_state}) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] time.sleep(sleep_time) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return f(*args, **kwargs) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return self._wait_for_job(job_path) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] error_desc=err_desc) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During handling of the above exception, another exception occurred: 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Traceback (most recent call last): 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] yield resources 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] accel_info=accel_info) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(*args, **kwargs) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] admin_password, network_info, block_device_info) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(self, *args, **kwds) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] should_plug_vifs=False) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'req_state': req_state}) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] time.sleep(sleep_time) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return f(*args, **kwargs) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return self._wait_for_job(job_path) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] error_desc=err_desc) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.141 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.141 4076 3051113162536 MainThread INFO nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Terminating instance 2021-09-10 17:15:53.141 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:15:53.141 4076 3051113162536 MainThread INFO nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Got request to destroy instance 2021-09-10 17:15:53.167 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:15:53.167 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:53.167 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000072" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:53.167 4076 3051113162536 MainThread INFO nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:15:53.167 4076 3051113162536 MainThread DEBUG nova.compute.claims [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:15:53.167 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:53.359 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.hostops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:15:53.380 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.hostops [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:15:53.582 4076 3051113162536 MainThread DEBUG nova.compute.provider_tree [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:15:53.609 4076 3051113162536 MainThread DEBUG nova.scheduler.client.report [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:15:53.609 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:53.609 4076 3051113162536 MainThread ERROR nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Traceback (most recent call last): 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(*args, **kwargs) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] admin_password, network_info, block_device_info) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(self, *args, **kwds) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] should_plug_vifs=False) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'req_state': req_state}) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] time.sleep(sleep_time) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return f(*args, **kwargs) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return self._wait_for_job(job_path) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] error_desc=err_desc) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] During handling of the above exception, another exception occurred: 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Traceback (most recent call last): 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] accel_info=accel_info) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(*args, **kwargs) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] admin_password, network_info, block_device_info) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return function(self, *args, **kwds) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] should_plug_vifs=False) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'req_state': req_state}) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] time.sleep(sleep_time) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self.force_reraise() 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] raise self.value 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return f(*args, **kwargs) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] return self._wait_for_job(job_path) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] error_desc=err_desc) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. 2021-09-10 17:15:53.609 4076 3051113162536 MainThread TRACE nova.compute.manager [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.compute.utils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Build of instance 236a5f41-195e-4bd9-9c70-230e5a1cf40f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000072' failed to start. Not enough memory in the system to start the virtual machine instance-00000072. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000072' failed to start. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Not enough memory in the system to start the virtual machine instance-00000072 with ram size 1024 megabytes. (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) 'instance-00000072' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID A6279449-6D43-447B-922B-A1A3259238CB) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:15:53.620 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:15:54.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:15:54.797 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:54.797 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:54.797 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:15:54.797 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:15:54.818 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:15:54.960 4076 3051113162536 MainThread DEBUG nova.network.neutron [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:15:55.034 4076 3051113162536 MainThread INFO nova.compute.manager [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 236a5f41-195e-4bd9-9c70-230e5a1cf40f] Took 1.41 seconds to deallocate network for instance. 2021-09-10 17:15:55.034 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2221MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:15:55.034 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:15:55.304 4076 3051113162536 MainThread INFO nova.scheduler.client.report [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Deleted allocations for instance 236a5f41-195e-4bd9-9c70-230e5a1cf40f 2021-09-10 17:15:55.352 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-0d2d8f84-0a81-4f83-8b71-ac603395fb34 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "236a5f41-195e-4bd9-9c70-230e5a1cf40f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.047s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.352 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:15:55.375 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:15:55.375 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:15:55.416 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:15:55.437 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:15:55.638 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:15:55.656 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:15:55.656 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:15:55.656 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:25.822 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:25.859 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:16:26.037 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:26.037 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:16:26.037 4076 3051113164648 MainThread INFO nova.compute.claims [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Claim successful on node n-h1-808474-1 2021-09-10 17:16:26.265 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:26.271 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:26.475 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:16:26.511 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:16:26.511 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:26.511 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:16:26.631 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:16:26.631 4076 3051113161304 GreenThread-103 DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:16:26.631 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:16:26.646 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:16:26.846 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:16:26.865 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Spawning new instance 2021-09-10 17:16:26.865 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:26.865 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:26.881 4076 3051113164648 MainThread DEBUG nova.block_device [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:16:26.881 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:26.881 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:26.881 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Creating directory: C:\openstack\instances\instance-00000073 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:16:26.881 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000073\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:16:26.911 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Resizing VHD C:\openstack\instances\instance-00000073\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:16:27.047 4076 3051113161304 GreenThread-103 DEBUG nova.policy [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0f9bbe6baa01415c8d1cbf1560a0a6a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b8755ff27ce54a87aae58455dcebfc3c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:16:29.818 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Successfully created port: 69ea82b2-f141-467e-97ca-95e5a4503b0d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:16:32.737 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Successfully updated port: 69ea82b2-f141-467e-97ca-95e5a4503b0d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:16:32.755 4076 3051113161304 GreenThread-103 DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Acquired lock "refresh_cache-8ee5dab1-c5ca-4888-b656-e2359a016d43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:16:32.755 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:16:32.834 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Received event network-changed-69ea82b2-f141-467e-97ca-95e5a4503b0d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:16:32.834 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Refreshing instance network info cache due to event network-changed-69ea82b2-f141-467e-97ca-95e5a4503b0d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:16:33.375 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:16:34.755 4076 3051113161304 GreenThread-103 DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Updating instance_info_cache with network_info: [{"id": "69ea82b2-f141-467e-97ca-95e5a4503b0d", "address": "fa:16:3e:79:6b:3b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.23", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap69ea82b2-f1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:34.819 4076 3051113161304 GreenThread-103 DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Releasing lock "refresh_cache-8ee5dab1-c5ca-4888-b656-e2359a016d43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:16:34.819 4076 3051113161304 GreenThread-103 DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Instance network_info: |[{"id": "69ea82b2-f141-467e-97ca-95e5a4503b0d", "address": "fa:16:3e:79:6b:3b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.23", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap69ea82b2-f1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:16:34.819 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-8ee5dab1-c5ca-4888-b656-e2359a016d43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:16:34.819 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Refreshing network info cache for port 69ea82b2-f141-467e-97ca-95e5a4503b0d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:16:34.819 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Preparing to wait for external event network-vif-plugged-69ea82b2-f141-467e-97ca-95e5a4503b0d prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:16:34.819 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:34.819 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:34.827 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Creating VM instance-00000073 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:16:35.691 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:16:37.030 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Updated VIF entry in instance network info cache for port 69ea82b2-f141-467e-97ca-95e5a4503b0d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:16:37.030 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Updating instance_info_cache with network_info: [{"id": "69ea82b2-f141-467e-97ca-95e5a4503b0d", "address": "fa:16:3e:79:6b:3b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.23", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap69ea82b2-f1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:37.090 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-372f0a77-a797-47bd-a75f-3f8bd46148dc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-8ee5dab1-c5ca-4888-b656-e2359a016d43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:16:41.677 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:41.677 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:16:42.036 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:16:42.036 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:16:43.155 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-6231a7e2-01be-4176-9d1f-cb5244040ebe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Received event network-vif-plugged-69ea82b2-f141-467e-97ca-95e5a4503b0d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:16:43.155 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-6231a7e2-01be-4176-9d1f-cb5244040ebe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:43.155 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-6231a7e2-01be-4176-9d1f-cb5244040ebe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:43.155 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-6231a7e2-01be-4176-9d1f-cb5244040ebe 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Processing event network-vif-plugged-69ea82b2-f141-467e-97ca-95e5a4503b0d _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:16:43.189 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Using config drive for instance 2021-09-10 17:16:43.307 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:43.319 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:16:43.319 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:16:43.319 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:43.389 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Creating config drive at C:\openstack\instances\instance-00000073\configdrive.iso 2021-09-10 17:16:43.429 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000073\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpc9e5hgmh execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:16:43.735 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000073\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpc9e5hgmh" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:16:44.141 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Setting boot order for instance: instance-00000073: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:16:44.188 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:16:44.211 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '521D4860-7200-4681-9B21-57E1824D382D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15016), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:44.347 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:44.347 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:44.347 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:44.347 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:44.347 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '521D4860-7200-4681-9B21-57E1824D382D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113117), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:44.347 4076 3051113164648 MainThread DEBUG os_win._utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1586439609527588 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:16:44.364 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-12d7be17-fbe4-4634-abb1-d92f8ff321bd - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:44.364 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-12d7be17-fbe4-4634-abb1-d92f8ff321bd - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:44.431 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-12d7be17-fbe4-4634-abb1-d92f8ff321bd - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:44.458 4076 3051113163768 MainThread INFO nova.compute.manager [req-12d7be17-fbe4-4634-abb1-d92f8ff321bd - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:45.380 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '3FF00725-E06B-48B5-A91F-EA0610728686', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18608), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:45.514 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:45.514 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:45.514 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:45.514 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:45.530 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3FF00725-E06B-48B5-A91F-EA0610728686', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110194), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:45.530 4076 3051113164648 MainThread DEBUG os_win._utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3423218727111816 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:16:45.530 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-d8c9eeb3-6c2a-4f7b-a6bd-f2f17af85554 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:45.530 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-d8c9eeb3-6c2a-4f7b-a6bd-f2f17af85554 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:45.596 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-d8c9eeb3-6c2a-4f7b-a6bd-f2f17af85554 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:45.642 4076 3051113163768 MainThread INFO nova.compute.manager [req-d8c9eeb3-6c2a-4f7b-a6bd-f2f17af85554 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:46.562 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': 'A98A9ECE-2243-4FD4-9833-86FCDABC6636', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15769), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:46.674 4076 3051113163768 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:46.674 4076 3051113163768 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:46.674 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:46.701 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:46.703 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A98A9ECE-2243-4FD4-9833-86FCDABC6636', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110712), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:46.703 4076 3051113164648 MainThread DEBUG os_win._utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5146985054016113 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:16:46.724 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0d93ddfe-71f7-4ef1-834d-47c11407133e - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:46.724 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [req-0d93ddfe-71f7-4ef1-834d-47c11407133e - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:46.770 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-0d93ddfe-71f7-4ef1-834d-47c11407133e - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:46.802 4076 3051113163768 MainThread INFO nova.compute.manager [req-0d93ddfe-71f7-4ef1-834d-47c11407133e - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:46.832 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:46.862 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:16:47.161 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:47.161 4076 3051113163064 MainThread DEBUG nova.virt.hardware [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:16:47.161 4076 3051113163064 MainThread INFO nova.compute.claims [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Claim successful on node n-h1-808474-1 2021-09-10 17:16:47.452 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:47.452 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:47.661 4076 3051113163064 MainThread DEBUG nova.compute.provider_tree [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:16:47.687 4076 3051113163064 MainThread DEBUG nova.scheduler.client.report [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:16:47.687 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:47.687 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:16:47.740 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '1B5871EC-A517-4C1B-B470-07ECDCFDE0DD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14828), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:47.831 4076 3051113163064 MainThread DEBUG nova.compute.utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:16:47.831 4076 3051113163768 GreenThread-104 DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:16:47.843 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:16:47.867 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:16:47.867 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:47.876 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:47.876 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:47.876 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:47.893 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1B5871EC-A517-4C1B-B470-07ECDCFDE0DD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111579), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:47.893 4076 3051113164648 MainThread DEBUG os_win._utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7046947479248047 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:16:48.051 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-d565dc67-02c3-4b15-9da3-a3f1684f7576 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:48.051 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-d565dc67-02c3-4b15-9da3-a3f1684f7576 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:48.115 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-d565dc67-02c3-4b15-9da3-a3f1684f7576 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:48.328 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:16:48.343 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Spawning new instance 2021-09-10 17:16:48.343 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG nova.block_device [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG os_win.utils.pathutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Creating directory: C:\openstack\instances\instance-00000074 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:16:48.364 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000074\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:16:48.378 4076 3051113161304 MainThread INFO nova.compute.manager [req-d565dc67-02c3-4b15-9da3-a3f1684f7576 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:48.393 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Resizing VHD C:\openstack\instances\instance-00000074\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:16:48.926 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '8FDEF7E2-713F-4F86-BE34-FCB79ED90492', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15266), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:49.052 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:49.052 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:49.052 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:49.052 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:49.067 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8FDEF7E2-713F-4F86-BE34-FCB79ED90492', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110872), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:49.067 4076 3051113164648 MainThread DEBUG os_win._utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.879055023193359 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:16:49.083 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-ec399a51-5714-4a7a-9747-c068932c7e27 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:49.083 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-ec399a51-5714-4a7a-9747-c068932c7e27 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:49.129 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-ec399a51-5714-4a7a-9747-c068932c7e27 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:49.176 4076 3051113161304 MainThread INFO nova.compute.manager [req-ec399a51-5714-4a7a-9747-c068932c7e27 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:49.271 4076 3051113163768 GreenThread-104 DEBUG nova.policy [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '11e4c6eac8924ee882931a301b388a54', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '12cf0aeb647e41119e2189d6819df350', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:16:50.079 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '2F3F637F-C06C-413C-9D6C-6076F2161195', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14988), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:50.225 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:50.225 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:50.225 4076 3051113161304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:16:50.225 4076 3051113161304 MainThread INFO nova.compute.manager [-] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] VM Stopped (Lifecycle Event) 2021-09-10 17:16:50.237 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000073\' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C519810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000073\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)instance-00000073F999AB3E-5367-417F-BB17-7D877E4C5198%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2F3F637F-C06C-413C-9D6C-6076F2161195', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111067), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 16, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 16, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\nNot enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)\n\n'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198)", 'ErrorSummaryDescription': "'instance-00000073' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000073.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:16:50.237 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Failed to change vm state of instance-00000073 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. 2021-09-10 17:16:50.237 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Got request to destroy instance 2021-09-10 17:16:50.251 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-ff73ae40-e38b-4a1b-a027-b6b1ea9fbcd2 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:16:50.251 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [req-ff73ae40-e38b-4a1b-a027-b6b1ea9fbcd2 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:16:50.301 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-ff73ae40-e38b-4a1b-a027-b6b1ea9fbcd2 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:16:50.333 4076 3051113161304 MainThread INFO nova.compute.manager [req-ff73ae40-e38b-4a1b-a027-b6b1ea9fbcd2 - - - - -] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:16:50.389 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:50.764 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:51.581 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Successfully created port: 3357a684-0937-49f4-8ae8-0acf6575fb2d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:16:51.746 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:51.896 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:51.911 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:51.911 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:16:52.036 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:16:52.036 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:52.036 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:52.052 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Successfully changed vm state of instance-00000073 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:16:52.052 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Successfully changed state of VM instance-00000073 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:16:52.100 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Waiting for WMI job: {'InstanceID': '579E3E3B-F24F-48E3-96A2-B39EDFC8D8E0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15038), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:16:52.224 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] WMI job succeeded: {'InstanceID': '579E3E3B-F24F-48E3-96A2-B39EDFC8D8E0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29626), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:16:52.224 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:52.224 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Removing directory: C:\openstack\instances\instance-00000073 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:16:52.224 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:52.224 4076 3051113164648 MainThread ERROR nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Traceback (most recent call last): 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(*args, **kwargs) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] admin_password, network_info, block_device_info) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(self, *args, **kwds) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] should_plug_vifs=False) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'req_state': req_state}) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] time.sleep(sleep_time) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return f(*args, **kwargs) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return self._wait_for_job(job_path) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] error_desc=err_desc) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During handling of the above exception, another exception occurred: 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Traceback (most recent call last): 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] yield resources 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] accel_info=accel_info) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(*args, **kwargs) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] admin_password, network_info, block_device_info) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(self, *args, **kwds) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] should_plug_vifs=False) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'req_state': req_state}) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] time.sleep(sleep_time) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return f(*args, **kwargs) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return self._wait_for_job(job_path) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] error_desc=err_desc) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:52.224 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:52.224 4076 3051113164648 MainThread INFO nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Terminating instance 2021-09-10 17:16:52.224 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:16:52.224 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Got request to destroy instance 2021-09-10 17:16:52.249 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:16:52.249 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:52.249 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "instance-00000073" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:52.249 4076 3051113164648 MainThread INFO nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:16:52.264 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:16:52.264 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:52.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:52.880 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:16:52.943 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:16:52.943 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:16:52.985 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:16:53.098 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:16:53.098 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:53.109 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:53.317 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:16:53.333 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:16:53.333 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 1.063s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:53.333 4076 3051113164648 MainThread ERROR nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Traceback (most recent call last): 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(*args, **kwargs) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] admin_password, network_info, block_device_info) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(self, *args, **kwds) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] should_plug_vifs=False) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'req_state': req_state}) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] time.sleep(sleep_time) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return f(*args, **kwargs) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return self._wait_for_job(job_path) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] error_desc=err_desc) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] During handling of the above exception, another exception occurred: 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Traceback (most recent call last): 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] accel_info=accel_info) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(*args, **kwargs) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] admin_password, network_info, block_device_info) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return function(self, *args, **kwds) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] should_plug_vifs=False) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'req_state': req_state}) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] time.sleep(sleep_time) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self.force_reraise() 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] raise self.value 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return f(*args, **kwargs) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] return self._wait_for_job(job_path) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] error_desc=err_desc) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. 2021-09-10 17:16:53.333 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] 2021-09-10 17:16:53.333 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:16:53.333 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Build of instance 8ee5dab1-c5ca-4888-b656-e2359a016d43 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000073' failed to start. Not enough memory in the system to start the virtual machine instance-00000073. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000073' failed to start. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Not enough memory in the system to start the virtual machine instance-00000073 with ram size 1024 megabytes. (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) 'instance-00000073' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID F999AB3E-5367-417F-BB17-7D877E4C5198) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:16:53.346 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:16:53.346 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:16:53.346 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:16:53.346 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:16:53.764 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:53.780 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Successfully updated port: 3357a684-0937-49f4-8ae8-0acf6575fb2d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:16:53.815 4076 3051113163768 GreenThread-104 DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Acquired lock "refresh_cache-93488f38-5aea-4764-bdcd-f98205b2fdbf" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:16:53.815 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:16:53.905 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Received event network-changed-3357a684-0937-49f4-8ae8-0acf6575fb2d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:16:53.905 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Refreshing instance network info cache due to event network-changed-3357a684-0937-49f4-8ae8-0acf6575fb2d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:16:54.505 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:16:54.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:54.813 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:54.813 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:54.813 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:16:54.813 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:54.813 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:55.039 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2218MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:16:55.039 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:55.311 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.327 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 8ee5dab1-c5ca-4888-b656-e2359a016d43 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 17:16:55.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 93488f38-5aea-4764-bdcd-f98205b2fdbf actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:16:55.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:16:55.353 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:16:55.420 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:55.436 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:55.499 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:55.551 4076 3051113164648 MainThread INFO nova.compute.manager [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] [instance: 8ee5dab1-c5ca-4888-b656-e2359a016d43] Took 2.20 seconds to deallocate network for instance. 2021-09-10 17:16:55.651 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:16:55.661 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:16:55.661 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:16:55.661 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.609s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:55.875 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Deleted allocations for instance 8ee5dab1-c5ca-4888-b656-e2359a016d43 2021-09-10 17:16:55.895 4076 3051113163768 GreenThread-104 DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Updating instance_info_cache with network_info: [{"id": "3357a684-0937-49f4-8ae8-0acf6575fb2d", "address": "fa:16:3e:7d:c5:67", "network": {"id": "532751df-3933-413f-b51b-1349c0db7399", "bridge": null, "label": "tempest-VolumesActionsTest-94880735-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "12cf0aeb647e41119e2189d6819df350", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3357a684-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:55.911 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ffcf8c73-ce46-468c-be2e-59a434f1ea2d 0f9bbe6baa01415c8d1cbf1560a0a6a3 b8755ff27ce54a87aae58455dcebfc3c - default default] Lock "8ee5dab1-c5ca-4888-b656-e2359a016d43" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:55.952 4076 3051113163768 GreenThread-104 DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Releasing lock "refresh_cache-93488f38-5aea-4764-bdcd-f98205b2fdbf" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:16:55.973 4076 3051113163768 GreenThread-104 DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Instance network_info: |[{"id": "3357a684-0937-49f4-8ae8-0acf6575fb2d", "address": "fa:16:3e:7d:c5:67", "network": {"id": "532751df-3933-413f-b51b-1349c0db7399", "bridge": null, "label": "tempest-VolumesActionsTest-94880735-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "12cf0aeb647e41119e2189d6819df350", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3357a684-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:16:55.973 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Preparing to wait for external event network-vif-plugged-3357a684-0937-49f4-8ae8-0acf6575fb2d prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:16:55.973 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:55.973 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:55.973 4076 3051113163064 MainThread DEBUG os_win.utils.compute.vmutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Creating VM instance-00000074 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:16:55.973 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-93488f38-5aea-4764-bdcd-f98205b2fdbf" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:16:55.973 4076 3051055218008 MainThread DEBUG nova.network.neutron [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Refreshing network info cache for port 3357a684-0937-49f4-8ae8-0acf6575fb2d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:16:56.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:56.848 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:16:57.691 4076 3051055218008 MainThread DEBUG nova.network.neutron [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Updated VIF entry in instance network info cache for port 3357a684-0937-49f4-8ae8-0acf6575fb2d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:16:57.691 4076 3051055218008 MainThread DEBUG nova.network.neutron [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Updating instance_info_cache with network_info: [{"id": "3357a684-0937-49f4-8ae8-0acf6575fb2d", "address": "fa:16:3e:7d:c5:67", "network": {"id": "532751df-3933-413f-b51b-1349c0db7399", "bridge": null, "label": "tempest-VolumesActionsTest-94880735-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "12cf0aeb647e41119e2189d6819df350", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap3357a684-09", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:16:57.717 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:57.724 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [req-8b63e46d-0ab3-4aff-9c93-8952b4d2c990 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-93488f38-5aea-4764-bdcd-f98205b2fdbf" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:16:57.749 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:16:57.811 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:16:57.811 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:16:57.896 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:57.896 4076 3051113161128 MainThread DEBUG nova.virt.hardware [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:16:57.896 4076 3051113161128 MainThread INFO nova.compute.claims [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Claim successful on node n-h1-808474-1 2021-09-10 17:16:58.098 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:16:58.114 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:16:58.317 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:16:58.332 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:16:58.332 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.438s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:58.345 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:16:58.437 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:16:58.437 4076 3051113161480 GreenThread-105 DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:16:58.437 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:16:58.473 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:16:58.614 4076 3051113161480 GreenThread-105 DEBUG nova.policy [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '799e4a49be0e448b9d6d54d57b7d8d1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cf483b0ac5349bca4c898bb3b806d21', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:16:58.689 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:16:58.710 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Spawning new instance 2021-09-10 17:16:58.724 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:58.724 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:58.724 4076 3051113161128 MainThread DEBUG nova.block_device [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:16:58.724 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:16:58.724 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:16:58.734 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating directory: C:\openstack\instances\instance-00000075 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:16:58.734 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000075\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:16:58.770 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Resizing VHD C:\openstack\instances\instance-00000075\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:16:59.745 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Successfully created port: de1feead-a6f3-41b0-a15d-f97530739043 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:17:01.455 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Successfully updated port: de1feead-a6f3-41b0-a15d-f97530739043 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:17:01.484 4076 3051113161480 GreenThread-105 DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Acquired lock "refresh_cache-3b484a16-f305-421b-a31a-69bd404d1d4c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:17:01.484 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:17:01.677 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:17:02.530 4076 3051113161480 GreenThread-105 DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Updating instance_info_cache with network_info: [{"id": "de1feead-a6f3-41b0-a15d-f97530739043", "address": "fa:16:3e:0a:a2:90", "network": {"id": "4fcf38f4-2fb6-40c8-b7df-43fa2b0eb6ff", "bridge": null, "label": "tempest-network-smoke--1532811086", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapde1feead-a6", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:17:02.577 4076 3051113161480 GreenThread-105 DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Releasing lock "refresh_cache-3b484a16-f305-421b-a31a-69bd404d1d4c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:17:02.577 4076 3051113161480 GreenThread-105 DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Instance network_info: |[{"id": "de1feead-a6f3-41b0-a15d-f97530739043", "address": "fa:16:3e:0a:a2:90", "network": {"id": "4fcf38f4-2fb6-40c8-b7df-43fa2b0eb6ff", "bridge": null, "label": "tempest-network-smoke--1532811086", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapde1feead-a6", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:17:02.577 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Preparing to wait for external event network-vif-plugged-de1feead-a6f3-41b0-a15d-f97530739043 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:17:02.577 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:02.577 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:02.577 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating VM instance-00000075 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:17:03.254 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Received event network-changed-de1feead-a6f3-41b0-a15d-f97530739043 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:17:03.254 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Refreshing instance network info cache due to event network-changed-de1feead-a6f3-41b0-a15d-f97530739043. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:17:03.254 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-3b484a16-f305-421b-a31a-69bd404d1d4c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:17:03.254 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Refreshing network info cache for port de1feead-a6f3-41b0-a15d-f97530739043 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:17:03.474 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:17:04.254 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Updated VIF entry in instance network info cache for port de1feead-a6f3-41b0-a15d-f97530739043. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:17:04.254 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Updating instance_info_cache with network_info: [{"id": "de1feead-a6f3-41b0-a15d-f97530739043", "address": "fa:16:3e:0a:a2:90", "network": {"id": "4fcf38f4-2fb6-40c8-b7df-43fa2b0eb6ff", "bridge": null, "label": "tempest-network-smoke--1532811086", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapde1feead-a6", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:17:04.286 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-40aa12bf-8056-4cdc-89bd-51c89bd90bc3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-3b484a16-f305-421b-a31a-69bd404d1d4c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:17:05.005 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-66326183-2d1d-4e23-9b70-5ec125df3dcf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Received event network-vif-plugged-3357a684-0937-49f4-8ae8-0acf6575fb2d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:17:05.005 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-66326183-2d1d-4e23-9b70-5ec125df3dcf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:05.026 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-66326183-2d1d-4e23-9b70-5ec125df3dcf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:05.026 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-66326183-2d1d-4e23-9b70-5ec125df3dcf 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Processing event network-vif-plugged-3357a684-0937-49f4-8ae8-0acf6575fb2d _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:17:05.054 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Using config drive for instance 2021-09-10 17:17:05.235 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Creating config drive at C:\openstack\instances\instance-00000074\configdrive.iso 2021-09-10 17:17:05.249 4076 3051113163064 MainThread DEBUG oslo_concurrency.processutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000074\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpjmfs07wg execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:17:05.552 4076 3051113163064 MainThread DEBUG oslo_concurrency.processutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000074\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpjmfs07wg" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:17:05.952 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Setting boot order for instance: instance-00000074: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:17:06.014 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:17:06.042 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '0B4226AF-796B-4710-B8AE-3DE982C1F6E7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14587), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:06.182 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:06.182 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:06.182 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:06.191 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:06.191 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0B4226AF-796B-4710-B8AE-3DE982C1F6E7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114904), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:06.191 4076 3051113163064 MainThread DEBUG os_win._utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17720270156860352 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:06.243 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d7870c5a-eda6-44eb-bde8-69846fe9b347 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:06.243 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-d7870c5a-eda6-44eb-bde8-69846fe9b347 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:06.285 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d7870c5a-eda6-44eb-bde8-69846fe9b347 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:06.347 4076 3051113162008 MainThread INFO nova.compute.manager [req-d7870c5a-eda6-44eb-bde8-69846fe9b347 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:07.224 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '915017CF-F7B8-4C84-BD20-CBB86E03F19B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16183), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:07.346 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:07.346 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:07.346 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:07.346 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:07.363 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '915017CF-F7B8-4C84-BD20-CBB86E03F19B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111586), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:07.363 4076 3051113163064 MainThread DEBUG os_win._utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.349074125289917 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:07.380 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-41f21c99-ec9e-4ffb-9740-8e81d97b1b92 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:07.380 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-41f21c99-ec9e-4ffb-9740-8e81d97b1b92 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:07.421 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-41f21c99-ec9e-4ffb-9740-8e81d97b1b92 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:07.460 4076 3051113162008 MainThread INFO nova.compute.manager [req-41f21c99-ec9e-4ffb-9740-8e81d97b1b92 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:07.795 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:07.795 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:17:07.802 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:17:08.391 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '48142C95-8ADF-4164-93B5-459BCE7EA8CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15787), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:08.532 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:08.532 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:08.532 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:08.532 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:08.532 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '48142C95-8ADF-4164-93B5-459BCE7EA8CE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110042), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:08.532 4076 3051113163064 MainThread DEBUG os_win._utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5177130699157715 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:08.594 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-f1972b1e-6e02-499b-9cf2-af339203c84d - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:08.594 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-f1972b1e-6e02-499b-9cf2-af339203c84d - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:08.655 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-f1972b1e-6e02-499b-9cf2-af339203c84d - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:08.673 4076 3051113162008 MainThread INFO nova.compute.manager [req-f1972b1e-6e02-499b-9cf2-af339203c84d - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:09.565 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '7B36A801-9DB9-4FD8-A8F9-F5C31CC1FE3B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14933), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:09.696 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:09.696 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:09.708 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:09.708 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:09.708 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7B36A801-9DB9-4FD8-A8F9-F5C31CC1FE3B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112669), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:09.708 4076 3051113163064 MainThread DEBUG os_win._utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6942312717437744 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:09.750 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-04ed7759-bee2-4e22-949c-3fbc8f76eb44 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:09.750 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-04ed7759-bee2-4e22-949c-3fbc8f76eb44 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:09.795 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-04ed7759-bee2-4e22-949c-3fbc8f76eb44 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:09.833 4076 3051113162008 MainThread INFO nova.compute.manager [req-04ed7759-bee2-4e22-949c-3fbc8f76eb44 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:10.719 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': 'BFC2B562-D212-4E06-8242-A6A7C5E3F83A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15195), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:10.861 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:10.861 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:10.861 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:10.861 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:10.861 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BFC2B562-D212-4E06-8242-A6A7C5E3F83A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110131), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:10.861 4076 3051113163064 MainThread DEBUG os_win._utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.846842288970947 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:10.890 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-7d0d354f-68ea-4084-9cf2-5939f3574bd0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Received event network-vif-plugged-de1feead-a6f3-41b0-a15d-f97530739043 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:17:10.890 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-7d0d354f-68ea-4084-9cf2-5939f3574bd0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:10.890 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-7d0d354f-68ea-4084-9cf2-5939f3574bd0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:10.890 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-7d0d354f-68ea-4084-9cf2-5939f3574bd0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Processing event network-vif-plugged-de1feead-a6f3-41b0-a15d-f97530739043 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:17:10.907 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-b97e3d31-a790-4661-ad13-918a03bd4d67 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:10.907 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-b97e3d31-a790-4661-ad13-918a03bd4d67 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:10.932 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Using config drive for instance 2021-09-10 17:17:10.967 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-b97e3d31-a790-4661-ad13-918a03bd4d67 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:11.024 4076 3051113162008 MainThread INFO nova.compute.manager [req-b97e3d31-a790-4661-ad13-918a03bd4d67 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:11.126 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Creating config drive at C:\openstack\instances\instance-00000075\configdrive.iso 2021-09-10 17:17:11.186 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000075\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkmqqypl7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:17:11.474 4076 3051113161128 MainThread DEBUG oslo_concurrency.processutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000075\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkmqqypl7" returned: 0 in 0.282s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:17:11.894 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Setting boot order for instance: instance-00000075: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:17:11.913 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '2015A60F-A91A-4701-BB98-4C5A5D094DE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15572), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:11.942 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:17:11.972 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '8222A2B9-E045-412D-871D-C7AA84DF427B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14497), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:12.051 4076 3051113161480 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:12.051 4076 3051113161480 MainThread INFO nova.compute.manager [-] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] VM Stopped (Lifecycle Event) 2021-09-10 17:17:12.051 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:12.066 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:12.066 4076 3051113163064 MainThread ERROR os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000074\' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E512030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E510243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000074\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)instance-000000741EBFFAE6-488D-4D2B-B46C-52AF18D068E5%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2015A60F-A91A-4701-BB98-4C5A5D094DE2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=119159), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\nNot enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)\n\n'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5)", 'ErrorSummaryDescription': "'instance-00000074' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000074.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:12.066 4076 3051113163064 MainThread ERROR nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Failed to change vm state of instance-00000074 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. 2021-09-10 17:17:12.066 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Got request to destroy instance 2021-09-10 17:17:12.082 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-2a305167-b3ee-4760-9635-7bb27efab418 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:12.082 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [req-2a305167-b3ee-4760-9635-7bb27efab418 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:12.130 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-2a305167-b3ee-4760-9635-7bb27efab418 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:12.170 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:12.170 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:12.170 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:12.170 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:12.170 4076 3051113161480 MainThread INFO nova.compute.manager [req-2a305167-b3ee-4760-9635-7bb27efab418 - - - - -] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:12.200 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-7164c750-2f34-4107-b962-f6e03ea9912e - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:12.200 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-7164c750-2f34-4107-b962-f6e03ea9912e - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:12.235 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8222A2B9-E045-412D-871D-C7AA84DF427B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=171145), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:12.235 4076 3051113161128 MainThread DEBUG os_win._utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.29299378395080566 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:12.235 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-7164c750-2f34-4107-b962-f6e03ea9912e - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:12.279 4076 3051113164648 MainThread INFO nova.compute.manager [req-7164c750-2f34-4107-b962-f6e03ea9912e - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:13.265 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'FBE224DC-ADB6-4554-A43B-9BCB2AB1834D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15208), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:13.410 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:13.410 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:13.410 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:13.410 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:13.426 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FBE224DC-ADB6-4554-A43B-9BCB2AB1834D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114526), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:13.426 4076 3051113161128 MainThread DEBUG os_win._utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.4837822914123535 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:13.437 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-f2c7125c-7061-4d6d-ab82-b48de1ed90ca - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:13.437 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-f2c7125c-7061-4d6d-ab82-b48de1ed90ca - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:13.487 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-f2c7125c-7061-4d6d-ab82-b48de1ed90ca - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:13.535 4076 3051113164648 MainThread INFO nova.compute.manager [req-f2c7125c-7061-4d6d-ab82-b48de1ed90ca - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:13.905 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:17:13.905 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:13.905 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:13.926 4076 3051113163064 MainThread DEBUG os_win.utils.compute.vmutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Successfully changed vm state of instance-00000074 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:17:13.926 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Successfully changed state of VM instance-00000074 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:17:13.968 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Waiting for WMI job: {'InstanceID': '3A95319C-9128-4D73-AC45-FEB41A225887', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18784), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:14.077 4076 3051113163064 MainThread DEBUG os_win.utils.jobutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] WMI job succeeded: {'InstanceID': '3A95319C-9128-4D73-AC45-FEB41A225887', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30648), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:17:14.077 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:14.077 4076 3051113163064 MainThread DEBUG os_win.utils.pathutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Removing directory: C:\openstack\instances\instance-00000074 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:17:14.077 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:14.077 4076 3051113163064 MainThread ERROR nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Traceback (most recent call last): 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(*args, **kwargs) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] admin_password, network_info, block_device_info) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(self, *args, **kwds) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] should_plug_vifs=False) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'req_state': req_state}) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] time.sleep(sleep_time) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return f(*args, **kwargs) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return self._wait_for_job(job_path) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] error_desc=err_desc) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During handling of the above exception, another exception occurred: 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Traceback (most recent call last): 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] yield resources 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] accel_info=accel_info) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(*args, **kwargs) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] admin_password, network_info, block_device_info) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(self, *args, **kwds) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] should_plug_vifs=False) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'req_state': req_state}) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] time.sleep(sleep_time) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return f(*args, **kwargs) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return self._wait_for_job(job_path) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] error_desc=err_desc) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.077 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.077 4076 3051113163064 MainThread INFO nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Terminating instance 2021-09-10 17:17:14.099 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:17:14.099 4076 3051113163064 MainThread INFO nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Got request to destroy instance 2021-09-10 17:17:14.116 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:17:14.116 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:14.116 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "instance-00000074" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:14.116 4076 3051113163064 MainThread INFO nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:17:14.116 4076 3051113163064 MainThread DEBUG nova.compute.claims [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:17:14.116 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:14.389 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:17:14.389 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.hostops [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:17:14.438 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '6087D609-5992-4D42-B2EE-3D11235FA255', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16881), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:14.594 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:14.594 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:14.594 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:14.594 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:14.655 4076 3051113163064 MainThread DEBUG nova.compute.provider_tree [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.scheduler.client.report [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.563s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:14.673 4076 3051113163064 MainThread ERROR nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Traceback (most recent call last): 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(*args, **kwargs) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] admin_password, network_info, block_device_info) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(self, *args, **kwds) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] should_plug_vifs=False) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'req_state': req_state}) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] time.sleep(sleep_time) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return f(*args, **kwargs) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return self._wait_for_job(job_path) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] error_desc=err_desc) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] During handling of the above exception, another exception occurred: 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Traceback (most recent call last): 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] accel_info=accel_info) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(*args, **kwargs) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] admin_password, network_info, block_device_info) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return function(self, *args, **kwds) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] should_plug_vifs=False) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'req_state': req_state}) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] time.sleep(sleep_time) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self.force_reraise() 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] raise self.value 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return f(*args, **kwargs) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] return self._wait_for_job(job_path) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] error_desc=err_desc) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. 2021-09-10 17:17:14.673 4076 3051113163064 MainThread TRACE nova.compute.manager [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.compute.utils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Build of instance 93488f38-5aea-4764-bdcd-f98205b2fdbf was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000074' failed to start. Not enough memory in the system to start the virtual machine instance-00000074. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000074' failed to start. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Not enough memory in the system to start the virtual machine instance-00000074 with ram size 1024 megabytes. (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) 'instance-00000074' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1EBFFAE6-488D-4D2B-B46C-52AF18D068E5) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:17:14.673 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:17:14.692 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-70ffbb3c-8d35-4177-b226-3122b4b878ea - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:14.692 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-70ffbb3c-8d35-4177-b226-3122b4b878ea - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:14.692 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6087D609-5992-4D42-B2EE-3D11235FA255', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=137814), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:14.692 4076 3051113161128 MainThread DEBUG os_win._utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.750030040740967 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:14.739 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-70ffbb3c-8d35-4177-b226-3122b4b878ea - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:14.751 4076 3051113164648 MainThread INFO nova.compute.manager [req-70ffbb3c-8d35-4177-b226-3122b4b878ea - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:15.708 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '98BCCCE8-BAD1-4815-888E-181690BCAF8A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15673), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:15.843 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:15.843 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:15.843 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:15.843 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:15.843 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '98BCCCE8-BAD1-4815-888E-181690BCAF8A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111899), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:15.843 4076 3051113161128 MainThread DEBUG os_win._utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.9005837440490723 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:15.880 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-2b5d20f4-4753-4c19-a240-a9e6c454d1ac - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:15.880 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-2b5d20f4-4753-4c19-a240-a9e6c454d1ac - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:15.924 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-2b5d20f4-4753-4c19-a240-a9e6c454d1ac - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:15.998 4076 3051055216776 MainThread INFO nova.compute.manager [req-2b5d20f4-4753-4c19-a240-a9e6c454d1ac - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:16.879 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '814E251C-2B26-45A4-8D43-A8AF3B7A1361', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14984), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:17.015 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:17.015 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:17.015 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:17.015 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:17.015 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '814E251C-2B26-45A4-8D43-A8AF3B7A1361', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114924), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:17.015 4076 3051113161128 MainThread DEBUG os_win._utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.072536468505859 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:17:17.053 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-0a1f51a5-480b-42a4-8c07-bdeeee4892af - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:17.053 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-0a1f51a5-480b-42a4-8c07-bdeeee4892af - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:17.093 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-0a1f51a5-480b-42a4-8c07-bdeeee4892af - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:17.130 4076 3051055216776 MainThread INFO nova.compute.manager [req-0a1f51a5-480b-42a4-8c07-bdeeee4892af - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:17.350 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:17:17.390 4076 3051113163064 MainThread INFO nova.compute.manager [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] [instance: 93488f38-5aea-4764-bdcd-f98205b2fdbf] Took 2.72 seconds to deallocate network for instance. 2021-09-10 17:17:17.872 4076 3051113163064 MainThread INFO nova.scheduler.client.report [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Deleted allocations for instance 93488f38-5aea-4764-bdcd-f98205b2fdbf 2021-09-10 17:17:17.973 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-4de6e491-143a-4d9f-91fd-b9b8fb7252fe 11e4c6eac8924ee882931a301b388a54 12cf0aeb647e41119e2189d6819df350 - default default] Lock "93488f38-5aea-4764-bdcd-f98205b2fdbf" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 31.141s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:18.037 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '671346E7-DD45-4A6D-8828-92170670978E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14713), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:18.172 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:18.172 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:18.172 4076 3051113163064 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:17:18.172 4076 3051113163064 MainThread INFO nova.compute.manager [-] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] VM Stopped (Lifecycle Event) 2021-09-10 17:17:18.188 4076 3051113161128 MainThread ERROR os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000075\' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F2691210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000075\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)instance-00000075BA1EFC02-DE71-4891-B0A9-2C65F9F26912%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '671346E7-DD45-4A6D-8828-92170670978E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116809), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 17, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 17, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\nNot enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)\n\n'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912)", 'ErrorSummaryDescription': "'instance-00000075' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000075.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:17:18.188 4076 3051113161128 MainThread ERROR nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Failed to change vm state of instance-00000075 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. 2021-09-10 17:17:18.188 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Got request to destroy instance 2021-09-10 17:17:18.251 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-8655a0c2-f9ee-49a1-b305-58e22f75370f - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:17:18.251 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [req-8655a0c2-f9ee-49a1-b305-58e22f75370f - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:17:18.296 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-8655a0c2-f9ee-49a1-b305-58e22f75370f - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:17:18.350 4076 3051113163064 MainThread INFO nova.compute.manager [req-8655a0c2-f9ee-49a1-b305-58e22f75370f - - - - -] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:17:19.973 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:17:19.973 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:19.973 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:19.998 4076 3051113161128 MainThread DEBUG os_win.utils.compute.vmutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed vm state of instance-00000075 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:17:19.998 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed state of VM instance-00000075 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:17:20.029 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '5E17027B-8D84-4D94-B992-6C388C976137', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13747), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:17:20.154 4076 3051113161128 MainThread DEBUG os_win.utils.jobutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job succeeded: {'InstanceID': '5E17027B-8D84-4D94-B992-6C388C976137', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28153), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:17:20.154 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:20.154 4076 3051113161128 MainThread DEBUG os_win.utils.pathutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Removing directory: C:\openstack\instances\instance-00000075 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:17:20.154 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:20.154 4076 3051113161128 MainThread ERROR nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Traceback (most recent call last): 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(*args, **kwargs) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] admin_password, network_info, block_device_info) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(self, *args, **kwds) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] should_plug_vifs=False) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'req_state': req_state}) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] time.sleep(sleep_time) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return f(*args, **kwargs) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return self._wait_for_job(job_path) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] error_desc=err_desc) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During handling of the above exception, another exception occurred: 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Traceback (most recent call last): 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] yield resources 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] accel_info=accel_info) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(*args, **kwargs) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] admin_password, network_info, block_device_info) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(self, *args, **kwds) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] should_plug_vifs=False) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'req_state': req_state}) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] time.sleep(sleep_time) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return f(*args, **kwargs) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return self._wait_for_job(job_path) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] error_desc=err_desc) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.154 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.176 4076 3051113161128 MainThread INFO nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Terminating instance 2021-09-10 17:17:20.176 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:17:20.176 4076 3051113161128 MainThread INFO nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Got request to destroy instance 2021-09-10 17:17:20.194 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.vmops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:17:20.194 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:20.194 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000075" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:20.194 4076 3051113161128 MainThread INFO nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:17:20.194 4076 3051113161128 MainThread DEBUG nova.compute.claims [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:17:20.194 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:20.342 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:17:20.370 4076 3051113161128 MainThread DEBUG nova.virt.hyperv.hostops [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:17:20.576 4076 3051113161128 MainThread DEBUG nova.compute.provider_tree [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.scheduler.client.report [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:20.583 4076 3051113161128 MainThread ERROR nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Traceback (most recent call last): 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(*args, **kwargs) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] admin_password, network_info, block_device_info) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(self, *args, **kwds) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] should_plug_vifs=False) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'req_state': req_state}) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] time.sleep(sleep_time) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return f(*args, **kwargs) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return self._wait_for_job(job_path) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] error_desc=err_desc) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] During handling of the above exception, another exception occurred: 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Traceback (most recent call last): 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] accel_info=accel_info) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(*args, **kwargs) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] admin_password, network_info, block_device_info) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return function(self, *args, **kwds) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] should_plug_vifs=False) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'req_state': req_state}) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] time.sleep(sleep_time) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self.force_reraise() 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] raise self.value 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return f(*args, **kwargs) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] return self._wait_for_job(job_path) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] error_desc=err_desc) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. 2021-09-10 17:17:20.583 4076 3051113161128 MainThread TRACE nova.compute.manager [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.compute.utils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Build of instance 3b484a16-f305-421b-a31a-69bd404d1d4c was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000075' failed to start. Not enough memory in the system to start the virtual machine instance-00000075. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000075' failed to start. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Not enough memory in the system to start the virtual machine instance-00000075 with ram size 1024 megabytes. (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) 'instance-00000075' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BA1EFC02-DE71-4891-B0A9-2C65F9F26912) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:17:20.583 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:17:22.310 4076 3051113161128 MainThread DEBUG nova.network.neutron [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:17:22.342 4076 3051113161128 MainThread INFO nova.compute.manager [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 3b484a16-f305-421b-a31a-69bd404d1d4c] Took 1.77 seconds to deallocate network for instance. 2021-09-10 17:17:22.842 4076 3051113161128 MainThread INFO nova.scheduler.client.report [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Deleted allocations for instance 3b484a16-f305-421b-a31a-69bd404d1d4c 2021-09-10 17:17:22.889 4076 3051113161128 MainThread DEBUG oslo_concurrency.lockutils [req-a9ae6c12-f46e-4a0e-bf78-a165de5b40b2 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "3b484a16-f305-421b-a31a-69bd404d1d4c" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.171s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:40.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:40.767 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:17:40.989 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:17:40.989 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:17:42.291 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:17:42.322 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:17:42.322 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:17:43.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:49.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:50.982 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:51.797 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:51.797 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:51.813 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:51.813 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:17:52.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:55.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:56.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:17:56.801 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:56.801 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:17:56.801 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:17:56.801 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:17:56.814 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:17:57.028 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2214MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:17:57.044 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:17:57.239 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:17:57.300 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:17:57.300 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:17:57.499 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:17:57.531 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:17:57.531 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:17:57.531 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:12.731 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:12.933 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:18:12.935 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051055217480 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.935 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.935 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.953 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.953 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.953 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.953 4076 3051093170312 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.953 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.953 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:12.953 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:12.953 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:13.138 4076 3051055217480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.217 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.282s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.217 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.282s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.310 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.346 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.389 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.389 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.389 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.389 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.389 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.407 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:13.407 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:14.904 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:14.953 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:18:15.172 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:15.172 4076 3051055214664 MainThread DEBUG nova.virt.hardware [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:18:15.172 4076 3051055214664 MainThread INFO nova.compute.claims [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Claim successful on node n-h1-808474-1 2021-09-10 17:18:15.394 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:15.428 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:15.644 4076 3051055214664 MainThread DEBUG nova.compute.provider_tree [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:15.660 4076 3051055214664 MainThread DEBUG nova.scheduler.client.report [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:15.660 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:15.660 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:18:15.866 4076 3051055214664 MainThread DEBUG nova.compute.utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:18:15.866 4076 3051055217304 GreenThread-106 DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:18:15.866 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:18:15.895 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:18:16.129 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:18:16.153 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Spawning new instance 2021-09-10 17:18:16.169 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:16.169 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:16.169 4076 3051055214664 MainThread DEBUG nova.block_device [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:18:16.176 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:16.176 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:16.176 4076 3051055214664 MainThread DEBUG os_win.utils.pathutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating directory: C:\openstack\instances\instance-00000076 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:18:16.176 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000076\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:18:16.208 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Resizing VHD C:\openstack\instances\instance-00000076\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:18:16.285 4076 3051055217304 GreenThread-106 DEBUG nova.policy [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '799e4a49be0e448b9d6d54d57b7d8d1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cf483b0ac5349bca4c898bb3b806d21', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:18:17.861 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Successfully created port: bef65dd1-5158-4c1b-9844-609ac4a04f69 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:18:19.716 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Successfully updated port: bef65dd1-5158-4c1b-9844-609ac4a04f69 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:18:19.733 4076 3051055217304 GreenThread-106 DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Acquired lock "refresh_cache-2c0ae919-97f4-4b59-a070-63a3ec8effa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:19.733 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:18:19.856 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Received event network-changed-bef65dd1-5158-4c1b-9844-609ac4a04f69 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:19.856 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Refreshing instance network info cache due to event network-changed-bef65dd1-5158-4c1b-9844-609ac4a04f69. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:18:19.973 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:18:20.684 4076 3051055217304 GreenThread-106 DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Updating instance_info_cache with network_info: [{"id": "bef65dd1-5158-4c1b-9844-609ac4a04f69", "address": "fa:16:3e:62:73:a2", "network": {"id": "d0efb974-5cdf-4275-bab1-2abb154484f9", "bridge": null, "label": "tempest-network-smoke--1033545172", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbef65dd1-51", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:20.717 4076 3051055217304 GreenThread-106 DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Releasing lock "refresh_cache-2c0ae919-97f4-4b59-a070-63a3ec8effa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:20.717 4076 3051055217304 GreenThread-106 DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Instance network_info: |[{"id": "bef65dd1-5158-4c1b-9844-609ac4a04f69", "address": "fa:16:3e:62:73:a2", "network": {"id": "d0efb974-5cdf-4275-bab1-2abb154484f9", "bridge": null, "label": "tempest-network-smoke--1033545172", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbef65dd1-51", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:18:20.717 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-2c0ae919-97f4-4b59-a070-63a3ec8effa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:20.717 4076 3051049643704 MainThread DEBUG nova.network.neutron [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Refreshing network info cache for port bef65dd1-5158-4c1b-9844-609ac4a04f69 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:18:20.717 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Preparing to wait for external event network-vif-plugged-bef65dd1-5158-4c1b-9844-609ac4a04f69 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:18:20.717 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:20.717 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:20.717 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating VM instance-00000076 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:18:21.607 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:18:22.497 4076 3051049643704 MainThread DEBUG nova.network.neutron [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Updated VIF entry in instance network info cache for port bef65dd1-5158-4c1b-9844-609ac4a04f69. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:18:22.509 4076 3051049643704 MainThread DEBUG nova.network.neutron [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Updating instance_info_cache with network_info: [{"id": "bef65dd1-5158-4c1b-9844-609ac4a04f69", "address": "fa:16:3e:62:73:a2", "network": {"id": "d0efb974-5cdf-4275-bab1-2abb154484f9", "bridge": null, "label": "tempest-network-smoke--1033545172", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbef65dd1-51", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:22.518 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-00848471-552a-45dd-aedf-53e9dfed819f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-2c0ae919-97f4-4b59-a070-63a3ec8effa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:31.263 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-6aebb0bc-8ce3-4234-99fd-402b3c30cb02 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Received event network-vif-plugged-bef65dd1-5158-4c1b-9844-609ac4a04f69 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:31.263 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-6aebb0bc-8ce3-4234-99fd-402b3c30cb02 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:31.263 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-6aebb0bc-8ce3-4234-99fd-402b3c30cb02 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:31.263 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-6aebb0bc-8ce3-4234-99fd-402b3c30cb02 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Processing event network-vif-plugged-bef65dd1-5158-4c1b-9844-609ac4a04f69 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:18:31.309 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Using config drive for instance 2021-09-10 17:18:31.438 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Creating config drive at C:\openstack\instances\instance-00000076\configdrive.iso 2021-09-10 17:18:31.497 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000076\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpfvejkaut execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:18:31.647 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:31.668 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:18:31.809 4076 3051055214664 MainThread DEBUG oslo_concurrency.processutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000076\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpfvejkaut" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:18:31.819 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:31.819 4076 3051049643352 MainThread DEBUG nova.virt.hardware [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:18:31.819 4076 3051049643352 MainThread INFO nova.compute.claims [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Claim successful on node n-h1-808474-1 2021-09-10 17:18:32.060 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:32.080 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:32.203 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Setting boot order for instance: instance-00000076: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:18:32.253 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:18:32.296 4076 3051049643352 MainThread DEBUG nova.compute.provider_tree [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:32.296 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '27880361-EFE6-4390-AD6D-138C189A5A8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21279), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:32.313 4076 3051049643352 MainThread DEBUG nova.scheduler.client.report [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:32.313 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:32.313 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:18:32.413 4076 3051049643352 MainThread DEBUG nova.compute.utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:18:32.413 4076 3051055215720 GreenThread-107 DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:18:32.413 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:18:32.428 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:32.428 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:32.428 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:32.428 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:32.448 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:18:32.448 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '27880361-EFE6-4390-AD6D-138C189A5A8D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115632), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:32.448 4076 3051055214664 MainThread DEBUG os_win._utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19442462921142578 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:32.473 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-b1f1839d-71b9-4b23-956b-371bc9e54dca - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:32.473 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-b1f1839d-71b9-4b23-956b-371bc9e54dca - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:32.520 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-b1f1839d-71b9-4b23-956b-371bc9e54dca - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:32.543 4076 3051055216776 MainThread INFO nova.compute.manager [req-b1f1839d-71b9-4b23-956b-371bc9e54dca - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:32.629 4076 3051055215720 GreenThread-107 DEBUG nova.policy [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bb6b40339c39491bbe2bdee1c7c82123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fecf4120b310452eb14917a31b551102', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:18:32.629 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:18:32.650 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Spawning new instance 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG nova.block_device [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG os_win.utils.pathutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Creating directory: C:\openstack\instances\instance-00000077 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:18:32.662 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000077\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:18:32.700 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Resizing VHD C:\openstack\instances\instance-00000077\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:18:33.476 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '6CA8CA08-8D74-419A-9FAB-E5485D38048A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15122), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:33.612 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:33.612 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:33.612 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:33.612 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:33.624 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6CA8CA08-8D74-419A-9FAB-E5485D38048A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111407), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:33.624 4076 3051055214664 MainThread DEBUG os_win._utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3703246116638184 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:33.643 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-37de6500-c6fd-4382-90d5-42c84ccbec4c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:33.643 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-37de6500-c6fd-4382-90d5-42c84ccbec4c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:33.691 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-37de6500-c6fd-4382-90d5-42c84ccbec4c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:33.780 4076 3051055216776 MainThread INFO nova.compute.manager [req-37de6500-c6fd-4382-90d5-42c84ccbec4c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:33.957 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Successfully created port: 6ee0b4c5-c304-4ae5-832a-f243e7f44edb _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:18:34.653 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '3929333F-FEC3-4BCC-BDD8-69DA89E30417', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14605), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:34.769 4076 3051055216776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:34.769 4076 3051055216776 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:34.787 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:34.787 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:34.787 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3929333F-FEC3-4BCC-BDD8-69DA89E30417', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112511), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:34.787 4076 3051055214664 MainThread DEBUG os_win._utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.533536434173584 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:34.812 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-9bfed800-9a08-43d1-bad8-063a317f273c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:34.812 4076 3051055216776 MainThread DEBUG nova.virt.hyperv.vmops [req-9bfed800-9a08-43d1-bad8-063a317f273c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:34.856 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-9bfed800-9a08-43d1-bad8-063a317f273c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:34.890 4076 3051055216776 MainThread INFO nova.compute.manager [req-9bfed800-9a08-43d1-bad8-063a317f273c - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:35.092 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Successfully updated port: 6ee0b4c5-c304-4ae5-832a-f243e7f44edb _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:18:35.124 4076 3051055215720 GreenThread-107 DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Acquired lock "refresh_cache-27993f4e-58ab-4d61-a84c-7036ecfd1fad" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:35.124 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:18:35.233 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:18:35.348 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Received event network-changed-6ee0b4c5-c304-4ae5-832a-f243e7f44edb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:35.348 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Refreshing instance network info cache due to event network-changed-6ee0b4c5-c304-4ae5-832a-f243e7f44edb. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:18:35.825 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '4F64377E-A5F2-4119-889C-635C9860A627', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15393), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:35.899 4076 3051055215720 GreenThread-107 DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Updating instance_info_cache with network_info: [{"id": "6ee0b4c5-c304-4ae5-832a-f243e7f44edb", "address": "fa:16:3e:9e:ab:5a", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6ee0b4c5-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:35.910 4076 3051055215720 GreenThread-107 DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Releasing lock "refresh_cache-27993f4e-58ab-4d61-a84c-7036ecfd1fad" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:35.910 4076 3051055215720 GreenThread-107 DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Instance network_info: |[{"id": "6ee0b4c5-c304-4ae5-832a-f243e7f44edb", "address": "fa:16:3e:9e:ab:5a", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6ee0b4c5-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:18:35.910 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-27993f4e-58ab-4d61-a84c-7036ecfd1fad" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:35.910 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Refreshing network info cache for port 6ee0b4c5-c304-4ae5-832a-f243e7f44edb _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:18:35.931 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Preparing to wait for external event network-vif-plugged-6ee0b4c5-c304-4ae5-832a-f243e7f44edb prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:18:35.931 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:35.931 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:35.931 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Creating VM instance-00000077 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:18:35.966 4076 3051049643176 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:35.966 4076 3051049643176 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:35.966 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:35.966 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:35.966 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4F64377E-A5F2-4119-889C-635C9860A627', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113270), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:35.966 4076 3051055214664 MainThread DEBUG os_win._utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7128567695617676 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:35.990 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-92f93d52-5b6f-4301-a973-8a9b711e4af7 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:35.990 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-92f93d52-5b6f-4301-a973-8a9b711e4af7 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:36.045 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-92f93d52-5b6f-4301-a973-8a9b711e4af7 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:36.106 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:36.129 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:18:36.190 4076 3051049643176 MainThread INFO nova.compute.manager [req-92f93d52-5b6f-4301-a973-8a9b711e4af7 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:36.233 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:36.233 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:18:36.262 4076 3051113164648 MainThread INFO nova.compute.claims [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Claim successful on node n-h1-808474-1 2021-09-10 17:18:36.379 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:36.379 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:36.545 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Updated VIF entry in instance network info cache for port 6ee0b4c5-c304-4ae5-832a-f243e7f44edb. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:18:36.545 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Updating instance_info_cache with network_info: [{"id": "6ee0b4c5-c304-4ae5-832a-f243e7f44edb", "address": "fa:16:3e:9e:ab:5a", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap6ee0b4c5-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:36.577 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-1d6bdd59-b4c7-427a-82c3-d1232a0dd373 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-27993f4e-58ab-4d61-a84c-7036ecfd1fad" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:36.593 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:36.620 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:36.620 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:36.620 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:18:36.692 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:18:36.692 4076 3051049643176 GreenThread-108 DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:18:36.692 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:18:36.717 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:18:36.852 4076 3051049643176 GreenThread-108 DEBUG nova.policy [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '51b7f766d13c41e3b48015e63ff73fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fed35d747aa64087be13e1426708be67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:18:36.858 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:18:36.858 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:18:36.879 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Spawning new instance 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG nova.block_device [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Creating directory: C:\openstack\instances\instance-00000078 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:18:36.879 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000078\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:18:36.926 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Resizing VHD C:\openstack\instances\instance-00000078\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:18:37.013 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '9C0F8F5C-8CCA-4632-8032-AC983C76E4EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18214), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:37.138 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:37.138 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:37.138 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:37.156 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:37.156 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9C0F8F5C-8CCA-4632-8032-AC983C76E4EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117058), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:37.173 4076 3051055214664 MainThread DEBUG os_win._utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.919777154922485 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:37.173 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-4055a850-749e-4975-a75a-b9219cf47c72 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:37.173 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-4055a850-749e-4975-a75a-b9219cf47c72 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:37.216 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-4055a850-749e-4975-a75a-b9219cf47c72 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:37.247 4076 3051055217304 MainThread INFO nova.compute.manager [req-4055a850-749e-4975-a75a-b9219cf47c72 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:37.520 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Successfully created port: 4a031087-681a-49bb-90dc-ef633249a5a0 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:18:38.202 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '2E0883C6-833C-4055-BC78-C08DBC2B4916', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20282), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:38.323 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:38.323 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:38.323 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:38.323 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] VM Stopped (Lifecycle Event) 2021-09-10 17:18:38.348 4076 3051055214664 MainThread ERROR os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000076\' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000076\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)instance-000000761D9286AA-C560-4C51-A555-4D6C1FF5327E%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2E0883C6-833C-4055-BC78-C08DBC2B4916', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113376), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\nNot enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)\n\n'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E)", 'ErrorSummaryDescription': "'instance-00000076' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000076.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:38.348 4076 3051055214664 MainThread ERROR nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Failed to change vm state of instance-00000076 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. 2021-09-10 17:18:38.348 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Got request to destroy instance 2021-09-10 17:18:38.358 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-3b4e0a25-e899-412d-bdbc-3bcad8d98124 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:38.358 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-3b4e0a25-e899-412d-bdbc-3bcad8d98124 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:38.403 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-3b4e0a25-e899-412d-bdbc-3bcad8d98124 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:38.426 4076 3051055217304 MainThread INFO nova.compute.manager [req-3b4e0a25-e899-412d-bdbc-3bcad8d98124 - - - - -] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:38.426 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Successfully updated port: 4a031087-681a-49bb-90dc-ef633249a5a0 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:18:38.452 4076 3051049643176 GreenThread-108 DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Acquired lock "refresh_cache-145749dc-cbfd-4187-9e03-760c213b8b43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:38.452 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:18:38.538 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Received event network-changed-4a031087-681a-49bb-90dc-ef633249a5a0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:38.538 4076 3051055216776 MainThread DEBUG nova.compute.manager [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Refreshing instance network info cache due to event network-changed-4a031087-681a-49bb-90dc-ef633249a5a0. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:18:38.565 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:18:38.925 4076 3051049643176 GreenThread-108 DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Updating instance_info_cache with network_info: [{"id": "4a031087-681a-49bb-90dc-ef633249a5a0", "address": "fa:16:3e:92:76:0f", "network": {"id": "8080ea88-714a-417c-bc62-e1cbbcd08f8e", "bridge": null, "label": "tempest-TestServerBasicOps-227122155-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fed35d747aa64087be13e1426708be67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4a031087-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:38.944 4076 3051049643176 GreenThread-108 DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Releasing lock "refresh_cache-145749dc-cbfd-4187-9e03-760c213b8b43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:38.944 4076 3051049643176 GreenThread-108 DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Instance network_info: |[{"id": "4a031087-681a-49bb-90dc-ef633249a5a0", "address": "fa:16:3e:92:76:0f", "network": {"id": "8080ea88-714a-417c-bc62-e1cbbcd08f8e", "bridge": null, "label": "tempest-TestServerBasicOps-227122155-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fed35d747aa64087be13e1426708be67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4a031087-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:18:38.944 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-145749dc-cbfd-4187-9e03-760c213b8b43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:38.944 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Refreshing network info cache for port 4a031087-681a-49bb-90dc-ef633249a5a0 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:18:38.952 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Preparing to wait for external event network-vif-plugged-4a031087-681a-49bb-90dc-ef633249a5a0 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:18:38.952 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:38.952 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:38.952 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Creating VM instance-00000078 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:18:39.607 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Updated VIF entry in instance network info cache for port 4a031087-681a-49bb-90dc-ef633249a5a0. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:18:39.607 4076 3051055216776 MainThread DEBUG nova.network.neutron [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Updating instance_info_cache with network_info: [{"id": "4a031087-681a-49bb-90dc-ef633249a5a0", "address": "fa:16:3e:92:76:0f", "network": {"id": "8080ea88-714a-417c-bc62-e1cbbcd08f8e", "bridge": null, "label": "tempest-TestServerBasicOps-227122155-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "fed35d747aa64087be13e1426708be67", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4a031087-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:39.629 4076 3051055216776 MainThread DEBUG oslo_concurrency.lockutils [req-5eca9033-4f6d-4c74-ad48-dc7efa0c1dde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-145749dc-cbfd-4187-9e03-760c213b8b43" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:39.904 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:18:40.238 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:18:40.238 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:40.238 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:40.248 4076 3051055214664 MainThread DEBUG os_win.utils.compute.vmutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed vm state of instance-00000076 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:18:40.248 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed state of VM instance-00000076 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:18:40.310 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '66CC3B95-961B-430B-84D4-80092AC56780', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14093), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:40.421 4076 3051055214664 MainThread DEBUG os_win.utils.jobutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job succeeded: {'InstanceID': '66CC3B95-961B-430B-84D4-80092AC56780', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29059), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:18:40.421 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:40.421 4076 3051055214664 MainThread DEBUG os_win.utils.pathutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Removing directory: C:\openstack\instances\instance-00000076 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:18:40.421 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:40.421 4076 3051055214664 MainThread ERROR nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Traceback (most recent call last): 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(*args, **kwargs) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] admin_password, network_info, block_device_info) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(self, *args, **kwds) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] should_plug_vifs=False) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'req_state': req_state}) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] time.sleep(sleep_time) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return f(*args, **kwargs) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return self._wait_for_job(job_path) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] error_desc=err_desc) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During handling of the above exception, another exception occurred: 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Traceback (most recent call last): 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] yield resources 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] accel_info=accel_info) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(*args, **kwargs) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] admin_password, network_info, block_device_info) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(self, *args, **kwds) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] should_plug_vifs=False) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'req_state': req_state}) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] time.sleep(sleep_time) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return f(*args, **kwargs) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return self._wait_for_job(job_path) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] error_desc=err_desc) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.421 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.421 4076 3051055214664 MainThread INFO nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Terminating instance 2021-09-10 17:18:40.421 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:18:40.421 4076 3051055214664 MainThread INFO nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Got request to destroy instance 2021-09-10 17:18:40.465 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:18:40.465 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:40.465 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-00000076" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:40.465 4076 3051055214664 MainThread INFO nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:18:40.465 4076 3051055214664 MainThread DEBUG nova.compute.claims [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:18:40.465 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:40.613 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:40.613 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.hostops [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:40.826 4076 3051055214664 MainThread DEBUG nova.compute.provider_tree [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:40.843 4076 3051055214664 MainThread DEBUG nova.scheduler.client.report [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:40.843 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:40.843 4076 3051055214664 MainThread ERROR nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Traceback (most recent call last): 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(*args, **kwargs) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] admin_password, network_info, block_device_info) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(self, *args, **kwds) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] should_plug_vifs=False) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'req_state': req_state}) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] time.sleep(sleep_time) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return f(*args, **kwargs) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return self._wait_for_job(job_path) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] error_desc=err_desc) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] During handling of the above exception, another exception occurred: 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Traceback (most recent call last): 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] accel_info=accel_info) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(*args, **kwargs) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] admin_password, network_info, block_device_info) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return function(self, *args, **kwds) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] should_plug_vifs=False) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'req_state': req_state}) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] time.sleep(sleep_time) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self.force_reraise() 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] raise self.value 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return f(*args, **kwargs) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] return self._wait_for_job(job_path) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] error_desc=err_desc) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. 2021-09-10 17:18:40.843 4076 3051055214664 MainThread TRACE nova.compute.manager [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.compute.utils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Build of instance 2c0ae919-97f4-4b59-a070-63a3ec8effa7 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000076' failed to start. Not enough memory in the system to start the virtual machine instance-00000076. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000076' failed to start. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Not enough memory in the system to start the virtual machine instance-00000076 with ram size 1024 megabytes. (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) 'instance-00000076' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1D9286AA-C560-4C51-A555-4D6C1FF5327E) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:18:40.864 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:18:41.255 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-dfc0c72f-1cbe-4b27-8056-d75abcc6f7b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Received event network-vif-plugged-6ee0b4c5-c304-4ae5-832a-f243e7f44edb external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:41.255 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [req-dfc0c72f-1cbe-4b27-8056-d75abcc6f7b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:41.255 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [req-dfc0c72f-1cbe-4b27-8056-d75abcc6f7b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:41.255 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-dfc0c72f-1cbe-4b27-8056-d75abcc6f7b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Processing event network-vif-plugged-6ee0b4c5-c304-4ae5-832a-f243e7f44edb _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:18:41.324 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Using config drive for instance 2021-09-10 17:18:41.535 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Creating config drive at C:\openstack\instances\instance-00000077\configdrive.iso 2021-09-10 17:18:41.574 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000077\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl4dkkhc_ execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:18:41.735 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:41.748 4076 3051055214664 MainThread INFO nova.compute.manager [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2c0ae919-97f4-4b59-a070-63a3ec8effa7] Took 0.89 seconds to deallocate network for instance. 2021-09-10 17:18:41.856 4076 3051049643352 MainThread DEBUG oslo_concurrency.processutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000077\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpl4dkkhc_" returned: 0 in 0.281s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:18:41.952 4076 3051055214664 MainThread INFO nova.scheduler.client.report [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Deleted allocations for instance 2c0ae919-97f4-4b59-a070-63a3ec8effa7 2021-09-10 17:18:42.014 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-36968fc0-1598-4cdc-bbc9-8f30c9de52d9 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2c0ae919-97f4-4b59-a070-63a3ec8effa7" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.109s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:42.293 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Setting boot order for instance: instance-00000077: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:18:42.340 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:18:42.374 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': 'EF9B371D-F7CB-4BCC-89AD-9641AFD6A06C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14923), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:42.500 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:42.500 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:42.500 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:42.500 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:42.521 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EF9B371D-F7CB-4BCC-89AD-9641AFD6A06C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112275), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:42.521 4076 3051049643352 MainThread DEBUG os_win._utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18069720268249512 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:42.521 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d08ed443-b031-40e4-be1d-fcd6c18ca1fd - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:42.521 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-d08ed443-b031-40e4-be1d-fcd6c18ca1fd - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:42.564 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-d08ed443-b031-40e4-be1d-fcd6c18ca1fd - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:42.598 4076 3051055214664 MainThread INFO nova.compute.manager [req-d08ed443-b031-40e4-be1d-fcd6c18ca1fd - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:42.988 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:42.988 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:18:43.142 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:18:43.142 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:18:43.569 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '8D30ECDA-F9CF-4D22-A1D1-8BD069A4B1C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=20001), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:43.693 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:43.693 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:43.693 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:43.693 4076 3051093170312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:43.724 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-24597025-51fc-43fd-b884-6da6cb99b0a5 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:43.724 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-24597025-51fc-43fd-b884-6da6cb99b0a5 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:43.724 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8D30ECDA-F9CF-4D22-A1D1-8BD069A4B1C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113300), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:43.724 4076 3051049643352 MainThread DEBUG os_win._utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3833889961242676 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:43.768 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-24597025-51fc-43fd-b884-6da6cb99b0a5 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:43.793 4076 3051055215720 MainThread INFO nova.compute.manager [req-24597025-51fc-43fd-b884-6da6cb99b0a5 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:44.171 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:44.191 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:18:44.191 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:18:44.747 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '42993A43-5505-4DFC-A938-A1B90316884F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15063), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:44.873 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:44.873 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:44.873 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:44.873 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:44.896 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '42993A43-5505-4DFC-A938-A1B90316884F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112483), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:44.896 4076 3051049643352 MainThread DEBUG os_win._utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5557310581207275 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:44.910 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-cfc09d52-9d0c-434f-98da-b00000b08536 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:44.910 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-cfc09d52-9d0c-434f-98da-b00000b08536 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:44.941 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-cfc09d52-9d0c-434f-98da-b00000b08536 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:44.981 4076 3051055214664 MainThread INFO nova.compute.manager [req-cfc09d52-9d0c-434f-98da-b00000b08536 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:45.617 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-39cb0b9b-34b2-4ec2-96e2-9c3c5a6d25f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Received event network-vif-plugged-4a031087-681a-49bb-90dc-ef633249a5a0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:18:45.617 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-39cb0b9b-34b2-4ec2-96e2-9c3c5a6d25f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:45.624 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-39cb0b9b-34b2-4ec2-96e2-9c3c5a6d25f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:45.624 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-39cb0b9b-34b2-4ec2-96e2-9c3c5a6d25f1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Processing event network-vif-plugged-4a031087-681a-49bb-90dc-ef633249a5a0 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:18:45.660 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Using config drive for instance 2021-09-10 17:18:45.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:45.906 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Creating config drive at C:\openstack\instances\instance-00000078\configdrive.iso 2021-09-10 17:18:45.919 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000078\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpfwx5neuf execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:18:46.106 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '7F9C494D-9934-4EBE-AB7C-0FBDFC538A19', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14990), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:46.233 4076 3051093170488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:46.233 4076 3051093170488 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:46.233 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:46.233 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:46.254 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7F9C494D-9934-4EBE-AB7C-0FBDFC538A19', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113842), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:46.254 4076 3051049643352 MainThread DEBUG os_win._utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.914067029953003 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:46.269 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-69bdc723-513e-43db-bd70-7a0db4e89ce1 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:46.269 4076 3051093170488 MainThread DEBUG nova.virt.hyperv.vmops [req-69bdc723-513e-43db-bd70-7a0db4e89ce1 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:46.279 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000078\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpfwx5neuf" returned: 0 in 0.359s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:18:46.316 4076 3051093170488 MainThread DEBUG nova.compute.manager [req-69bdc723-513e-43db-bd70-7a0db4e89ce1 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:46.352 4076 3051093170488 MainThread INFO nova.compute.manager [req-69bdc723-513e-43db-bd70-7a0db4e89ce1 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:46.676 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Setting boot order for instance: instance-00000078: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:18:46.730 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:18:46.762 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': '99A138C3-82EB-43CD-9FBE-DE7B3AC1C682', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14771), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:46.890 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:46.890 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:46.890 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:46.890 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:46.890 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '99A138C3-82EB-43CD-9FBE-DE7B3AC1C682', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114967), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:46.890 4076 3051113164648 MainThread DEBUG os_win._utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1595141887664795 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:46.922 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c9e8f916-587c-41eb-bcca-6cc100dd3171 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:46.922 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-c9e8f916-587c-41eb-bcca-6cc100dd3171 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:46.963 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c9e8f916-587c-41eb-bcca-6cc100dd3171 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:46.988 4076 3051055214664 MainThread INFO nova.compute.manager [req-c9e8f916-587c-41eb-bcca-6cc100dd3171 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:47.271 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '498B7E6F-0BEF-4F46-A82B-0CB004FBCE96', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15092), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:47.412 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:47.412 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:47.412 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:47.412 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:47.412 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '498B7E6F-0BEF-4F46-A82B-0CB004FBCE96', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:47.412 4076 3051049643352 MainThread DEBUG os_win._utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.071473121643066 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:47.427 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-4a1bb137-7f48-4faa-94f4-619533ee72d7 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:47.427 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-4a1bb137-7f48-4faa-94f4-619533ee72d7 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:47.474 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-4a1bb137-7f48-4faa-94f4-619533ee72d7 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:47.501 4076 3051055214664 MainThread INFO nova.compute.manager [req-4a1bb137-7f48-4faa-94f4-619533ee72d7 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:47.908 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': '49E5AE0C-F0AF-4203-832C-ACDC22EFDB1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14630), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:48.041 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:48.045 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:48.045 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:48.045 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:48.045 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '49E5AE0C-F0AF-4203-832C-ACDC22EFDB1A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112001), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:48.045 4076 3051113164648 MainThread DEBUG os_win._utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3146302700042725 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:48.068 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-f83f1116-5fa4-4045-b2f5-2896bc18c0fc - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:48.068 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-f83f1116-5fa4-4045-b2f5-2896bc18c0fc - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:48.096 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-f83f1116-5fa4-4045-b2f5-2896bc18c0fc - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:48.130 4076 3051055214664 MainThread INFO nova.compute.manager [req-f83f1116-5fa4-4045-b2f5-2896bc18c0fc - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:48.441 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '6BAB4C04-7C13-4B85-8683-053743C70EE5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15080), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:48.574 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:48.578 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:48.578 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:48.578 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] VM Stopped (Lifecycle Event) 2021-09-10 17:18:48.578 4076 3051049643352 MainThread ERROR os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000077\' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000077\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)instance-000000771836D4E7-EC26-4699-955A-FB3B225ECBED%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6BAB4C04-7C13-4B85-8683-053743C70EE5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108644), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\nNot enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)\n\n'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED)", 'ErrorSummaryDescription': "'instance-00000077' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000077.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:48.578 4076 3051049643352 MainThread ERROR nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Failed to change vm state of instance-00000077 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. 2021-09-10 17:18:48.578 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Got request to destroy instance 2021-09-10 17:18:48.578 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-454c691d-ee69-4b06-ace7-f195f0a5dd39 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:48.578 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-454c691d-ee69-4b06-ace7-f195f0a5dd39 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:48.654 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-454c691d-ee69-4b06-ace7-f195f0a5dd39 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:48.677 4076 3051055214664 MainThread INFO nova.compute.manager [req-454c691d-ee69-4b06-ace7-f195f0a5dd39 - - - - -] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:49.075 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': '5A21EF5E-E962-4DC1-8E23-DA5B560F197C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15725), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:49.205 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:49.205 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:49.205 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:49.205 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:49.220 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5A21EF5E-E962-4DC1-8E23-DA5B560F197C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116668), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 49), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:49.220 4076 3051113164648 MainThread DEBUG os_win._utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.48901104927063 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:49.234 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-89c2791e-94d6-460d-97c6-73aeefcd6fdd - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:49.234 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-89c2791e-94d6-460d-97c6-73aeefcd6fdd - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:49.271 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-89c2791e-94d6-460d-97c6-73aeefcd6fdd - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:49.309 4076 3051055214664 MainThread INFO nova.compute.manager [req-89c2791e-94d6-460d-97c6-73aeefcd6fdd - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:50.262 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': '5BA6DFD4-6D5C-417D-B45B-9893D8A500D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15215), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:50.396 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:50.396 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:50.396 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:50.396 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:50.396 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5BA6DFD4-6D5C-417D-B45B-9893D8A500D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113713), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:50.419 4076 3051113164648 MainThread DEBUG os_win._utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6881399154663086 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:50.419 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:18:50.419 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:50.419 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:50.419 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ee27e1af-b556-43e5-bf48-41f7b2befa0b - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:50.419 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-ee27e1af-b556-43e5-bf48-41f7b2befa0b - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:50.419 4076 3051049643352 MainThread DEBUG os_win.utils.compute.vmutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Successfully changed vm state of instance-00000077 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:18:50.419 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Successfully changed state of VM instance-00000077 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:18:50.452 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-ee27e1af-b556-43e5-bf48-41f7b2befa0b - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:50.477 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '949F0DB2-896A-48CB-8999-BED498C752EB', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14278), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:50.477 4076 3051055214664 MainThread INFO nova.compute.manager [req-ee27e1af-b556-43e5-bf48-41f7b2befa0b - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread DEBUG os_win.utils.jobutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job succeeded: {'InstanceID': '949F0DB2-896A-48CB-8999-BED498C752EB', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28625), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:18:50.593 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:50.593 4076 3051049643352 MainThread DEBUG os_win.utils.pathutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Removing directory: C:\openstack\instances\instance-00000077 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:18:50.593 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:50.593 4076 3051049643352 MainThread ERROR nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Traceback (most recent call last): 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(*args, **kwargs) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] admin_password, network_info, block_device_info) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(self, *args, **kwds) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] should_plug_vifs=False) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'req_state': req_state}) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] time.sleep(sleep_time) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return f(*args, **kwargs) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return self._wait_for_job(job_path) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] error_desc=err_desc) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During handling of the above exception, another exception occurred: 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Traceback (most recent call last): 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] yield resources 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] accel_info=accel_info) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(*args, **kwargs) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] admin_password, network_info, block_device_info) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(self, *args, **kwds) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] should_plug_vifs=False) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'req_state': req_state}) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] time.sleep(sleep_time) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return f(*args, **kwargs) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return self._wait_for_job(job_path) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] error_desc=err_desc) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.593 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.593 4076 3051049643352 MainThread INFO nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Terminating instance 2021-09-10 17:18:50.593 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:18:50.593 4076 3051049643352 MainThread INFO nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Got request to destroy instance 2021-09-10 17:18:50.630 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:18:50.630 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:50.630 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-00000077" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:50.630 4076 3051049643352 MainThread INFO nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:18:50.630 4076 3051049643352 MainThread DEBUG nova.compute.claims [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:18:50.630 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:50.740 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:50.740 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.hostops [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:50.952 4076 3051049643352 MainThread DEBUG nova.compute.provider_tree [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:50.966 4076 3051049643352 MainThread DEBUG nova.scheduler.client.report [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:50.966 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.329s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:50.966 4076 3051049643352 MainThread ERROR nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Traceback (most recent call last): 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(*args, **kwargs) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] admin_password, network_info, block_device_info) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(self, *args, **kwds) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] should_plug_vifs=False) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'req_state': req_state}) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] time.sleep(sleep_time) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return f(*args, **kwargs) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return self._wait_for_job(job_path) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] error_desc=err_desc) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] During handling of the above exception, another exception occurred: 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Traceback (most recent call last): 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] accel_info=accel_info) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(*args, **kwargs) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] admin_password, network_info, block_device_info) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return function(self, *args, **kwds) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] should_plug_vifs=False) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'req_state': req_state}) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] time.sleep(sleep_time) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self.force_reraise() 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] raise self.value 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return f(*args, **kwargs) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] return self._wait_for_job(job_path) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] error_desc=err_desc) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. 2021-09-10 17:18:50.966 4076 3051049643352 MainThread TRACE nova.compute.manager [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] 2021-09-10 17:18:50.966 4076 3051049643352 MainThread DEBUG nova.compute.utils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:18:50.966 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Build of instance 27993f4e-58ab-4d61-a84c-7036ecfd1fad was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000077' failed to start. Not enough memory in the system to start the virtual machine instance-00000077. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000077' failed to start. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Not enough memory in the system to start the virtual machine instance-00000077 with ram size 1024 megabytes. (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) 'instance-00000077' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1836D4E7-EC26-4699-955A-FB3B225ECBED) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:18:50.985 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:18:50.985 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:18:50.985 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:18:50.985 4076 3051049643352 MainThread DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:18:51.433 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': '43933014-7F47-4A0E-831C-507E77A3C504', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14394), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:51.560 4076 3051049643352 MainThread DEBUG nova.network.neutron [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:51.586 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:51.586 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:51.586 4076 3051113293784 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:51.586 4076 3051113293784 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:51.599 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '43933014-7F47-4A0E-831C-507E77A3C504', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111928), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:51.599 4076 3051113164648 MainThread DEBUG os_win._utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.868469715118408 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:18:51.615 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-e9908cba-c35f-47f4-8727-a3389829c611 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:51.615 4076 3051113293784 MainThread DEBUG nova.virt.hyperv.vmops [req-e9908cba-c35f-47f4-8727-a3389829c611 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:51.615 4076 3051049643352 MainThread INFO nova.compute.manager [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 27993f4e-58ab-4d61-a84c-7036ecfd1fad] Took 0.64 seconds to deallocate network for instance. 2021-09-10 17:18:51.660 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-e9908cba-c35f-47f4-8727-a3389829c611 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:51.699 4076 3051113293784 MainThread INFO nova.compute.manager [req-e9908cba-c35f-47f4-8727-a3389829c611 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:51.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:51.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:51.830 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:51.830 4076 3051049643352 MainThread INFO nova.scheduler.client.report [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Deleted allocations for instance 27993f4e-58ab-4d61-a84c-7036ecfd1fad 2021-09-10 17:18:51.883 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-14cce35c-ce11-4c8b-b24b-c45a2ca8731e bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "27993f4e-58ab-4d61-a84c-7036ecfd1fad" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 20.234s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:52.614 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': 'BAE8F675-5FFF-4531-B721-D5D0CB3A4482', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14803), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:52.764 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:52.764 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:18:52.764 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:52.764 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:52.764 4076 3051049643352 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:18:52.764 4076 3051049643352 MainThread INFO nova.compute.manager [-] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] VM Stopped (Lifecycle Event) 2021-09-10 17:18:52.802 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-d8a3817c-0669-40fb-8748-ca12f57ef063 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:18:52.802 4076 3051049643352 MainThread DEBUG nova.virt.hyperv.vmops [req-d8a3817c-0669-40fb-8748-ca12f57ef063 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:18:52.840 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-d8a3817c-0669-40fb-8748-ca12f57ef063 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:18:52.861 4076 3051049643352 MainThread INFO nova.compute.manager [req-d8a3817c-0669-40fb-8748-ca12f57ef063 - - - - -] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:18:52.880 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000078\' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E86669910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000078\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)instance-000000788FAC8040-B269-4D40-95BF-A49F7E866699%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BAE8F675-5FFF-4531-B721-D5D0CB3A4482', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116224), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 18, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 18, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\nNot enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)\n\n'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699)", 'ErrorSummaryDescription': "'instance-00000078' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000078.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:18:52.880 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Failed to change vm state of instance-00000078 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. 2021-09-10 17:18:52.880 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Got request to destroy instance 2021-09-10 17:18:53.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:54.675 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:18:54.675 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:54.675 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:54.675 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Successfully changed vm state of instance-00000078 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:18:54.675 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Successfully changed state of VM instance-00000078 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:18:54.721 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Waiting for WMI job: {'InstanceID': 'A8ABA405-BC38-4FE0-A225-69A4C0D42D46', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14570), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:18:54.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:54.841 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] WMI job succeeded: {'InstanceID': 'A8ABA405-BC38-4FE0-A225-69A4C0D42D46', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28438), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:18:54.841 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:54.841 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Removing directory: C:\openstack\instances\instance-00000078 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:18:54.841 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:54.841 4076 3051113164648 MainThread ERROR nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Traceback (most recent call last): 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(*args, **kwargs) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] admin_password, network_info, block_device_info) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(self, *args, **kwds) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] should_plug_vifs=False) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'req_state': req_state}) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] time.sleep(sleep_time) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return f(*args, **kwargs) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return self._wait_for_job(job_path) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] error_desc=err_desc) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During handling of the above exception, another exception occurred: 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Traceback (most recent call last): 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] yield resources 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] accel_info=accel_info) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(*args, **kwargs) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] admin_password, network_info, block_device_info) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(self, *args, **kwds) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] should_plug_vifs=False) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'req_state': req_state}) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] time.sleep(sleep_time) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return f(*args, **kwargs) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return self._wait_for_job(job_path) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] error_desc=err_desc) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:54.841 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:54.841 4076 3051113164648 MainThread INFO nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Terminating instance 2021-09-10 17:18:54.841 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:18:54.841 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Got request to destroy instance 2021-09-10 17:18:54.864 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:18:54.864 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:54.864 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "instance-00000078" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:54.864 4076 3051113164648 MainThread INFO nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:18:54.864 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:18:54.864 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:54.984 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:55.018 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:55.206 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:55.230 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:55.230 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:55.230 4076 3051113164648 MainThread ERROR nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Traceback (most recent call last): 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(*args, **kwargs) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] admin_password, network_info, block_device_info) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(self, *args, **kwds) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] should_plug_vifs=False) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'req_state': req_state}) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] time.sleep(sleep_time) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return f(*args, **kwargs) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return self._wait_for_job(job_path) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] error_desc=err_desc) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] During handling of the above exception, another exception occurred: 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Traceback (most recent call last): 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] accel_info=accel_info) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(*args, **kwargs) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] admin_password, network_info, block_device_info) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return function(self, *args, **kwds) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] should_plug_vifs=False) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'req_state': req_state}) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] time.sleep(sleep_time) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self.force_reraise() 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] raise self.value 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return f(*args, **kwargs) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] return self._wait_for_job(job_path) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] error_desc=err_desc) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. 2021-09-10 17:18:55.230 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Build of instance 145749dc-cbfd-4187-9e03-760c213b8b43 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000078' failed to start. Not enough memory in the system to start the virtual machine instance-00000078. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000078' failed to start. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Not enough memory in the system to start the virtual machine instance-00000078 with ram size 1024 megabytes. (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) 'instance-00000078' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8FAC8040-B269-4D40-95BF-A49F7E866699) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:18:55.246 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:18:56.418 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:18:56.449 4076 3051113164648 MainThread INFO nova.compute.manager [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] [instance: 145749dc-cbfd-4187-9e03-760c213b8b43] Took 1.20 seconds to deallocate network for instance. 2021-09-10 17:18:56.672 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Deleted allocations for instance 145749dc-cbfd-4187-9e03-760c213b8b43 2021-09-10 17:18:56.710 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-910dbf53-f35c-443e-aa97-1a636be69903 51b7f766d13c41e3b48015e63ff73fa4 fed35d747aa64087be13e1426708be67 - default default] Lock "145749dc-cbfd-4187-9e03-760c213b8b43" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 20.610s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:56.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:18:56.788 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:56.788 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:56.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:18:56.788 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:56.788 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:56.996 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:18:56.996 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:18:57.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:18:57.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:18:57.232 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:18:57.252 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:18:57.451 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:18:57.471 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:18:57.471 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:18:57.471 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:18:58.480 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:22.646 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2af9b131-c85d-4418-810b-00a540718354" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:22.661 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:19:22.844 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:22.855 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:19:22.855 4076 3051113164648 MainThread INFO nova.compute.claims [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Claim successful on node n-h1-808474-1 2021-09-10 17:19:23.039 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:23.059 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:23.255 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:23.277 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:23.277 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:23.292 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:19:23.356 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:19:23.356 4076 3051049643352 GreenThread-109 DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:19:23.356 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:19:23.395 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:19:23.653 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:19:23.653 4076 3051049643352 GreenThread-109 DEBUG nova.policy [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '799e4a49be0e448b9d6d54d57b7d8d1b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5cf483b0ac5349bca4c898bb3b806d21', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:19:23.670 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Spawning new instance 2021-09-10 17:19:23.670 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:23.670 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:23.670 4076 3051113164648 MainThread DEBUG nova.block_device [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:19:23.684 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:23.684 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:23.684 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating directory: C:\openstack\instances\instance-0000007a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:19:23.684 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:19:23.699 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Resizing VHD C:\openstack\instances\instance-0000007a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:19:24.261 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:24.272 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:19:24.499 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:24.499 4076 3051113294664 MainThread DEBUG nova.virt.hardware [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:19:24.499 4076 3051113294664 MainThread INFO nova.compute.claims [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Claim successful on node n-h1-808474-1 2021-09-10 17:19:24.724 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.hostops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:24.739 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.hostops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:24.835 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:24.872 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:19:24.953 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Successfully created port: b07dc06e-4100-4550-9b1f-a39e69626fea _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:19:24.970 4076 3051113294664 MainThread DEBUG nova.compute.provider_tree [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:24.970 4076 3051113294664 MainThread DEBUG nova.scheduler.client.report [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:24.970 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:24.970 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:19:25.051 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:25.061 4076 3051113295544 MainThread DEBUG nova.virt.hardware [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:19:25.061 4076 3051113295544 MainThread INFO nova.compute.claims [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Claim successful on node n-h1-808474-1 2021-09-10 17:19:25.179 4076 3051113294664 MainThread DEBUG nova.compute.utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:19:25.192 4076 3051113292728 GreenThread-110 DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:19:25.192 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:19:25.219 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:19:25.323 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.hostops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:25.328 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.hostops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:25.474 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:19:25.489 4076 3051113294664 MainThread INFO nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Spawning new instance 2021-09-10 17:19:25.489 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:25.489 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:25.489 4076 3051113294664 MainThread DEBUG nova.block_device [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:19:25.489 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:25.506 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:25.506 4076 3051113294664 MainThread DEBUG os_win.utils.pathutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Creating directory: C:\openstack\instances\instance-0000007b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:19:25.506 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:19:25.528 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Resizing VHD C:\openstack\instances\instance-0000007b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:19:25.528 4076 3051113295544 MainThread DEBUG nova.compute.provider_tree [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:25.559 4076 3051113295544 MainThread DEBUG nova.scheduler.client.report [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:25.559 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:25.559 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:19:25.559 4076 3051113292728 GreenThread-110 DEBUG nova.policy [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bb6b40339c39491bbe2bdee1c7c82123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fecf4120b310452eb14917a31b551102', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:19:25.698 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:19:25.698 4076 3051113292904 GreenThread-111 DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:19:25.698 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:19:26.027 4076 3051113295544 MainThread INFO nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Spawning new instance 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG nova.block_device [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:26.027 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:26.058 4076 3051113295544 MainThread DEBUG os_win.utils.pathutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Creating directory: C:\openstack\instances\instance-00000079 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:19:26.058 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000079\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:19:26.075 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Resizing VHD C:\openstack\instances\instance-00000079\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:19:26.075 4076 3051113292904 GreenThread-111 DEBUG nova.policy [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ddf55d6b15144f37ae1d2f48a5918a49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be13a4a12834460091659ecf8dd9d41f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:19:26.693 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Successfully updated port: b07dc06e-4100-4550-9b1f-a39e69626fea _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:19:26.722 4076 3051049643352 GreenThread-109 DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Acquired lock "refresh_cache-2af9b131-c85d-4418-810b-00a540718354" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:26.722 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:19:26.792 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Received event network-changed-b07dc06e-4100-4550-9b1f-a39e69626fea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:26.792 4076 3051113291848 MainThread DEBUG nova.compute.manager [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Refreshing instance network info cache due to event network-changed-b07dc06e-4100-4550-9b1f-a39e69626fea. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:19:26.860 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:19:27.386 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Successfully created port: bb132475-c14e-464f-8a2b-7200545ce25d _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:19:28.042 4076 3051049643352 GreenThread-109 DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Updating instance_info_cache with network_info: [{"id": "b07dc06e-4100-4550-9b1f-a39e69626fea", "address": "fa:16:3e:d2:c2:92", "network": {"id": "c5eaeeeb-7369-44f7-81bc-5a26c24d5db5", "bridge": null, "label": "tempest-network-smoke--1195186798", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb07dc06e-41", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:28.067 4076 3051049643352 GreenThread-109 DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Releasing lock "refresh_cache-2af9b131-c85d-4418-810b-00a540718354" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:28.067 4076 3051049643352 GreenThread-109 DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Instance network_info: |[{"id": "b07dc06e-4100-4550-9b1f-a39e69626fea", "address": "fa:16:3e:d2:c2:92", "network": {"id": "c5eaeeeb-7369-44f7-81bc-5a26c24d5db5", "bridge": null, "label": "tempest-network-smoke--1195186798", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb07dc06e-41", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:19:28.067 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-2af9b131-c85d-4418-810b-00a540718354" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:28.067 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Refreshing network info cache for port b07dc06e-4100-4550-9b1f-a39e69626fea _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:19:28.067 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Preparing to wait for external event network-vif-plugged-b07dc06e-4100-4550-9b1f-a39e69626fea prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:19:28.067 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2af9b131-c85d-4418-810b-00a540718354-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:28.067 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2af9b131-c85d-4418-810b-00a540718354-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:28.067 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Creating VM instance-0000007a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:19:28.168 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Successfully created port: d6d665c6-3e84-4cc6-984d-be752b233887 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:19:28.952 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:19:29.268 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Successfully updated port: bb132475-c14e-464f-8a2b-7200545ce25d _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:19:29.293 4076 3051113292728 GreenThread-110 DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Acquired lock "refresh_cache-56af32cb-10b4-48fe-9adb-25b79d8d4df9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:29.293 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:19:29.433 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:19:29.463 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Updated VIF entry in instance network info cache for port b07dc06e-4100-4550-9b1f-a39e69626fea. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:19:29.463 4076 3051113291848 MainThread DEBUG nova.network.neutron [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Updating instance_info_cache with network_info: [{"id": "b07dc06e-4100-4550-9b1f-a39e69626fea", "address": "fa:16:3e:d2:c2:92", "network": {"id": "c5eaeeeb-7369-44f7-81bc-5a26c24d5db5", "bridge": null, "label": "tempest-network-smoke--1195186798", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "5cf483b0ac5349bca4c898bb3b806d21", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb07dc06e-41", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:29.480 4076 3051113291848 MainThread DEBUG oslo_concurrency.lockutils [req-f09bc888-ab07-48b8-92c7-b1a6bb31a66d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-2af9b131-c85d-4418-810b-00a540718354" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:29.817 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Received event network-changed-bb132475-c14e-464f-8a2b-7200545ce25d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:29.817 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Refreshing instance network info cache due to event network-changed-bb132475-c14e-464f-8a2b-7200545ce25d. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:19:29.817 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Successfully updated port: d6d665c6-3e84-4cc6-984d-be752b233887 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:19:29.847 4076 3051113292904 GreenThread-111 DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Acquired lock "refresh_cache-f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:29.847 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:19:29.904 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Received event network-changed-d6d665c6-3e84-4cc6-984d-be752b233887 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:29.904 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Refreshing instance network info cache due to event network-changed-d6d665c6-3e84-4cc6-984d-be752b233887. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:19:29.995 4076 3051113292728 GreenThread-110 DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Updating instance_info_cache with network_info: [{"id": "bb132475-c14e-464f-8a2b-7200545ce25d", "address": "fa:16:3e:86:f8:2c", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb132475-c1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:30.035 4076 3051113292728 GreenThread-110 DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Releasing lock "refresh_cache-56af32cb-10b4-48fe-9adb-25b79d8d4df9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:30.035 4076 3051113292728 GreenThread-110 DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Instance network_info: |[{"id": "bb132475-c14e-464f-8a2b-7200545ce25d", "address": "fa:16:3e:86:f8:2c", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb132475-c1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:19:30.035 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-56af32cb-10b4-48fe-9adb-25b79d8d4df9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:30.035 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Refreshing network info cache for port bb132475-c14e-464f-8a2b-7200545ce25d _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:19:30.035 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Preparing to wait for external event network-vif-plugged-bb132475-c14e-464f-8a2b-7200545ce25d prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:19:30.042 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:30.042 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:30.042 4076 3051113294664 MainThread DEBUG os_win.utils.compute.vmutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Creating VM instance-0000007b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:19:30.069 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:19:30.680 4076 3051113292904 GreenThread-111 DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Updating instance_info_cache with network_info: [{"id": "d6d665c6-3e84-4cc6-984d-be752b233887", "address": "fa:16:3e:cf:11:37", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd6d665c6-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:30.696 4076 3051113292904 GreenThread-111 DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Releasing lock "refresh_cache-f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:30.696 4076 3051113292904 GreenThread-111 DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Instance network_info: |[{"id": "d6d665c6-3e84-4cc6-984d-be752b233887", "address": "fa:16:3e:cf:11:37", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd6d665c6-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:19:30.708 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:30.708 4076 3051113294488 MainThread DEBUG nova.network.neutron [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Refreshing network info cache for port d6d665c6-3e84-4cc6-984d-be752b233887 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:19:30.708 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Preparing to wait for external event network-vif-plugged-d6d665c6-3e84-4cc6-984d-be752b233887 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:19:30.708 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:30.708 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:30.708 4076 3051113295544 MainThread DEBUG os_win.utils.compute.vmutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Creating VM instance-00000079 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:19:30.808 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Updated VIF entry in instance network info cache for port bb132475-c14e-464f-8a2b-7200545ce25d. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:19:30.808 4076 3051113160776 MainThread DEBUG nova.network.neutron [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Updating instance_info_cache with network_info: [{"id": "bb132475-c14e-464f-8a2b-7200545ce25d", "address": "fa:16:3e:86:f8:2c", "network": {"id": "e5325df2-5bda-4a32-9161-573dacd6e1e6", "bridge": null, "label": "tempest-VolumesSnapshotTestJSON-49926641-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fecf4120b310452eb14917a31b551102", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbb132475-c1", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:30.830 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-0086da01-5bec-445d-aadc-a1443b1045b3 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-56af32cb-10b4-48fe-9adb-25b79d8d4df9" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:30.971 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:19:31.647 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:19:32.106 4076 3051113294488 MainThread DEBUG nova.network.neutron [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Updated VIF entry in instance network info cache for port d6d665c6-3e84-4cc6-984d-be752b233887. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:19:32.106 4076 3051113294488 MainThread DEBUG nova.network.neutron [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Updating instance_info_cache with network_info: [{"id": "d6d665c6-3e84-4cc6-984d-be752b233887", "address": "fa:16:3e:cf:11:37", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd6d665c6-3e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:32.106 4076 3051113294488 MainThread DEBUG oslo_concurrency.lockutils [req-7c3d52fa-7c7c-4006-a3e7-e682b40246b0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:33.979 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-d782f938-3415-4f18-adb2-399501025b97 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Received event network-vif-plugged-b07dc06e-4100-4550-9b1f-a39e69626fea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:33.995 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-d782f938-3415-4f18-adb2-399501025b97 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2af9b131-c85d-4418-810b-00a540718354-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:33.995 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-d782f938-3415-4f18-adb2-399501025b97 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2af9b131-c85d-4418-810b-00a540718354-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:33.995 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-d782f938-3415-4f18-adb2-399501025b97 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Processing event network-vif-plugged-b07dc06e-4100-4550-9b1f-a39e69626fea _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:19:34.020 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Using config drive for instance 2021-09-10 17:19:34.196 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Creating config drive at C:\openstack\instances\instance-0000007a\configdrive.iso 2021-09-10 17:19:34.230 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpo5g0hwan execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:19:34.542 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpo5g0hwan" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:19:34.966 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Setting boot order for instance: instance-0000007a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:19:35.015 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:19:35.050 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '997BC309-F0A1-4C90-9BD6-419D50E1AF74', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19581), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:35.183 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:35.183 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 2af9b131-c85d-4418-810b-00a540718354] VM Stopped (Lifecycle Event) 2021-09-10 17:19:35.183 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:35.183 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:35.183 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '997BC309-F0A1-4C90-9BD6-419D50E1AF74', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116127), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:35.183 4076 3051113164648 MainThread DEBUG os_win._utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.168440580368042 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:35.200 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-77ac0bb7-5889-4b15-b2e3-4afe507ebc4d - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:35.200 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-77ac0bb7-5889-4b15-b2e3-4afe507ebc4d - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:35.254 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-77ac0bb7-5889-4b15-b2e3-4afe507ebc4d - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:35.285 4076 3051113160776 MainThread INFO nova.compute.manager [req-77ac0bb7-5889-4b15-b2e3-4afe507ebc4d - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:36.214 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'F261AF38-83D3-42D7-B148-087EDF9875CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15555), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:36.343 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:36.343 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 2af9b131-c85d-4418-810b-00a540718354] VM Stopped (Lifecycle Event) 2021-09-10 17:19:36.343 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:36.343 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:36.343 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F261AF38-83D3-42D7-B148-087EDF9875CF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113432), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:36.343 4076 3051113164648 MainThread DEBUG os_win._utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3277926445007324 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:36.377 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-b0178d50-be37-4e81-b462-50d95d94f49b - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:36.377 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-b0178d50-be37-4e81-b462-50d95d94f49b - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:36.417 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-b0178d50-be37-4e81-b462-50d95d94f49b - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:36.448 4076 3051113160776 MainThread INFO nova.compute.manager [req-b0178d50-be37-4e81-b462-50d95d94f49b - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:37.376 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '19092AB0-41FC-44F2-A551-3E9F8A39181C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16917), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:37.512 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:37.512 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:37.512 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:37.512 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 2af9b131-c85d-4418-810b-00a540718354] VM Stopped (Lifecycle Event) 2021-09-10 17:19:37.512 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '19092AB0-41FC-44F2-A551-3E9F8A39181C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111603), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:37.512 4076 3051113164648 MainThread DEBUG os_win._utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.497122049331665 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:37.558 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-46c2cee3-f80f-4f39-9dbb-41ab6576bc70 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:37.558 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-46c2cee3-f80f-4f39-9dbb-41ab6576bc70 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:37.577 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-d869126d-be74-4a15-80d8-9d882be1f9ad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Received event network-vif-plugged-bb132475-c14e-464f-8a2b-7200545ce25d external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:37.577 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-d869126d-be74-4a15-80d8-9d882be1f9ad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:37.577 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [req-d869126d-be74-4a15-80d8-9d882be1f9ad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:37.577 4076 3051049643352 MainThread DEBUG nova.compute.manager [req-d869126d-be74-4a15-80d8-9d882be1f9ad 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Processing event network-vif-plugged-bb132475-c14e-464f-8a2b-7200545ce25d _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:19:37.598 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-46c2cee3-f80f-4f39-9dbb-41ab6576bc70 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:37.609 4076 3051113294664 MainThread INFO nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Using config drive for instance 2021-09-10 17:19:37.640 4076 3051113160776 MainThread INFO nova.compute.manager [req-46c2cee3-f80f-4f39-9dbb-41ab6576bc70 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:37.895 4076 3051113294664 MainThread INFO nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Creating config drive at C:\openstack\instances\instance-0000007b\configdrive.iso 2021-09-10 17:19:37.919 4076 3051113294664 MainThread DEBUG oslo_concurrency.processutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp1l85dhnr execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:19:38.235 4076 3051113294664 MainThread DEBUG oslo_concurrency.processutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp1l85dhnr" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:19:38.543 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'DBD85C46-11C8-4E40-9B19-B42B2E909337', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15577), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:38.661 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Setting boot order for instance: instance-0000007b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:19:38.698 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DBD85C46-11C8-4E40-9B19-B42B2E909337', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112503), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:38.698 4076 3051113164648 MainThread DEBUG os_win._utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.682461977005005 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:38.715 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:19:38.745 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': 'E5647BE3-BCAE-4FC0-9CD6-DFE8605C9A8C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14471), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:38.877 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:38.877 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:38.877 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:38.892 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:38.892 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E5647BE3-BCAE-4FC0-9CD6-DFE8605C9A8C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111541), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:38.892 4076 3051113294664 MainThread DEBUG os_win._utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17657256126403809 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:38.908 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-82c9bf8b-1c41-4fc2-b383-357855e861da - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:38.908 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-82c9bf8b-1c41-4fc2-b383-357855e861da - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:38.950 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-82c9bf8b-1c41-4fc2-b383-357855e861da - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:38.979 4076 3051113160776 MainThread INFO nova.compute.manager [req-82c9bf8b-1c41-4fc2-b383-357855e861da - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:39.629 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-4ff049f6-90a4-4b4b-a641-71a89714c0ef 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Received event network-vif-plugged-d6d665c6-3e84-4cc6-984d-be752b233887 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:39.629 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-4ff049f6-90a4-4b4b-a641-71a89714c0ef 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:39.629 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [req-4ff049f6-90a4-4b4b-a641-71a89714c0ef 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:39.629 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-4ff049f6-90a4-4b4b-a641-71a89714c0ef 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Processing event network-vif-plugged-d6d665c6-3e84-4cc6-984d-be752b233887 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:19:39.663 4076 3051113295544 MainThread INFO nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Using config drive for instance 2021-09-10 17:19:39.749 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'C0661375-B181-494C-BDD6-266180B17F2D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15831), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:39.764 4076 3051113295544 MainThread INFO nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Creating config drive at C:\openstack\instances\instance-00000079\configdrive.iso 2021-09-10 17:19:39.797 4076 3051113295544 MainThread DEBUG oslo_concurrency.processutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000079\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmnhh70yg execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:19:39.971 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:39.971 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:39.971 4076 3051113294488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:39.971 4076 3051113294488 MainThread INFO nova.compute.manager [-] [instance: 2af9b131-c85d-4418-810b-00a540718354] VM Stopped (Lifecycle Event) 2021-09-10 17:19:39.980 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '11BEFA4A-415F-47C0-AE47-5A29DE25FCE1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17178), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:39.980 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C0661375-B181-494C-BDD6-266180B17F2D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113440), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:39.980 4076 3051113164648 MainThread DEBUG os_win._utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.965082406997681 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:39.997 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-a496edd9-2e9c-4ae4-9f79-5cd4b5993ab6 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:39.997 4076 3051113294488 MainThread DEBUG nova.virt.hyperv.vmops [req-a496edd9-2e9c-4ae4-9f79-5cd4b5993ab6 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:40.046 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-a496edd9-2e9c-4ae4-9f79-5cd4b5993ab6 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:40.067 4076 3051113294488 MainThread INFO nova.compute.manager [req-a496edd9-2e9c-4ae4-9f79-5cd4b5993ab6 - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:40.124 4076 3051113294488 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:40.124 4076 3051113294488 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:40.124 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:40.124 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:40.124 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '11BEFA4A-415F-47C0-AE47-5A29DE25FCE1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115258), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:40.124 4076 3051113294664 MainThread DEBUG os_win._utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.408585548400879 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:40.140 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-bb5ddd55-6850-41d1-99fc-044988eef2eb - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:40.140 4076 3051113294488 MainThread DEBUG nova.virt.hyperv.vmops [req-bb5ddd55-6850-41d1-99fc-044988eef2eb - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:40.140 4076 3051113295544 MainThread DEBUG oslo_concurrency.processutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000079\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmnhh70yg" returned: 0 in 0.343s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:19:40.192 4076 3051113294488 MainThread DEBUG nova.compute.manager [req-bb5ddd55-6850-41d1-99fc-044988eef2eb - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:40.228 4076 3051113294488 MainThread INFO nova.compute.manager [req-bb5ddd55-6850-41d1-99fc-044988eef2eb - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:40.558 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Setting boot order for instance: instance-00000079: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:19:40.610 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:19:40.645 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '77F8B28F-1FF4-48C4-BC0D-3225C78F7344', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15228), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:40.777 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:40.777 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:40.777 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:40.777 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:40.800 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-790c6dea-182c-4c3d-8d99-2895c10b0efc - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:40.800 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-790c6dea-182c-4c3d-8d99-2895c10b0efc - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:40.800 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '77F8B28F-1FF4-48C4-BC0D-3225C78F7344', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113457), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:40.817 4076 3051113295544 MainThread DEBUG os_win._utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.20740485191345215 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:40.839 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-790c6dea-182c-4c3d-8d99-2895c10b0efc - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:40.867 4076 3051113160776 MainThread INFO nova.compute.manager [req-790c6dea-182c-4c3d-8d99-2895c10b0efc - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:41.020 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': '3F84EC23-D839-41F0-8693-4ED3B9F3879C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14385), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:41.152 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:41.152 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 2af9b131-c85d-4418-810b-00a540718354] VM Stopped (Lifecycle Event) 2021-09-10 17:19:41.152 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:41.152 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:41.152 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': 'BDF9F1CE-586F-4C25-AD63-824823DB34BE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17369), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:41.173 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007a\' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)instance-0000007a84E0AF18-3017-461A-ADC5-630FC0CF5DE9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3F84EC23-D839-41F0-8693-4ED3B9F3879C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110767), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\nNot enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)\n\n'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9)", 'ErrorSummaryDescription': "'instance-0000007a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:41.173 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Failed to change vm state of instance-0000007a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. 2021-09-10 17:19:41.173 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Got request to destroy instance 2021-09-10 17:19:41.173 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-c991bc95-e970-4bd4-add4-f8c6b191acfe - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:41.173 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-c991bc95-e970-4bd4-add4-f8c6b191acfe - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:41.214 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-c991bc95-e970-4bd4-add4-f8c6b191acfe - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:41.236 4076 3051113160776 MainThread INFO nova.compute.manager [req-c991bc95-e970-4bd4-add4-f8c6b191acfe - - - - -] [instance: 2af9b131-c85d-4418-810b-00a540718354] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:41.295 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:41.295 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:41.295 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:41.295 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:41.295 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BDF9F1CE-586F-4C25-AD63-824823DB34BE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114772), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:41.295 4076 3051113294664 MainThread DEBUG os_win._utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.579425096511841 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:41.320 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-7e40daf1-3caa-45bb-b648-67a23bae21c5 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:41.320 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-7e40daf1-3caa-45bb-b648-67a23bae21c5 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:41.371 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-7e40daf1-3caa-45bb-b648-67a23bae21c5 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:41.395 4076 3051113160776 MainThread INFO nova.compute.manager [req-7e40daf1-3caa-45bb-b648-67a23bae21c5 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:41.867 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '66D528EF-1E72-4397-B599-B249B2BE2B48', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14980), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:41.999 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:41.999 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:41.999 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:41.999 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:41.999 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '66D528EF-1E72-4397-B599-B249B2BE2B48', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115542), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:41.999 4076 3051113295544 MainThread DEBUG os_win._utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3886663913726807 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:42.017 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-ea91cac7-5f7a-4b51-ab16-53ebcf8ba06d - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:42.017 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-ea91cac7-5f7a-4b51-ab16-53ebcf8ba06d - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:42.069 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-ea91cac7-5f7a-4b51-ab16-53ebcf8ba06d - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:42.086 4076 3051113160776 MainThread INFO nova.compute.manager [req-ea91cac7-5f7a-4b51-ab16-53ebcf8ba06d - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:42.298 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '9AFA87A4-F58F-4482-BD8B-CDD97543A0C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14693), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:42.450 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:42.450 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:42.450 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:42.450 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:42.474 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-59aff719-f608-42e9-8e83-910880008cf6 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:42.474 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-59aff719-f608-42e9-8e83-910880008cf6 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:42.519 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-59aff719-f608-42e9-8e83-910880008cf6 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:42.536 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9AFA87A4-F58F-4482-BD8B-CDD97543A0C6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117798), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:42.536 4076 3051113294664 MainThread DEBUG os_win._utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.8204851150512695 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:42.559 4076 3051113160776 MainThread INFO nova.compute.manager [req-59aff719-f608-42e9-8e83-910880008cf6 - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:43.017 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '8D8FEC10-9E44-4157-9324-1CA88189D21C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14947), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:43.060 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:19:43.060 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.060 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.073 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed vm state of instance-0000007a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:19:43.073 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Successfully changed state of VM instance-0000007a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:19:43.125 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Waiting for WMI job: {'InstanceID': 'E0E4543F-AEC5-4210-AEBF-A6585261C719', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=22533), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:43.141 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.141 4076 3051049643352 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.141 4076 3051113160776 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:43.141 4076 3051113160776 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:43.170 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8D8FEC10-9E44-4157-9324-1CA88189D21C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113069), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:43.170 4076 3051113295544 MainThread DEBUG os_win._utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5599067211151123 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:43.170 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-b0dc2119-a3b9-41f5-8c33-1b7aac93e96c - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:43.170 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [req-b0dc2119-a3b9-41f5-8c33-1b7aac93e96c - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:43.199 4076 3051113160776 MainThread DEBUG nova.compute.manager [req-b0dc2119-a3b9-41f5-8c33-1b7aac93e96c - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:43.229 4076 3051113160776 MainThread INFO nova.compute.manager [req-b0dc2119-a3b9-41f5-8c33-1b7aac93e96c - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:43.245 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] WMI job succeeded: {'InstanceID': 'E0E4543F-AEC5-4210-AEBF-A6585261C719', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=38825), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:19:43.245 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.245 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Removing directory: C:\openstack\instances\instance-0000007a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.248 4076 3051113164648 MainThread ERROR nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Traceback (most recent call last): 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(*args, **kwargs) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] admin_password, network_info, block_device_info) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(self, *args, **kwds) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] should_plug_vifs=False) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'req_state': req_state}) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] time.sleep(sleep_time) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return f(*args, **kwargs) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return self._wait_for_job(job_path) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] error_desc=err_desc) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] During handling of the above exception, another exception occurred: 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Traceback (most recent call last): 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] yield resources 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] accel_info=accel_info) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(*args, **kwargs) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] admin_password, network_info, block_device_info) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(self, *args, **kwds) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] should_plug_vifs=False) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'req_state': req_state}) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] time.sleep(sleep_time) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return f(*args, **kwargs) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return self._wait_for_job(job_path) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] error_desc=err_desc) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.248 4076 3051113164648 MainThread INFO nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Terminating instance 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:19:43.248 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Got request to destroy instance 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "instance-0000007a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.248 4076 3051113164648 MainThread INFO nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:19:43.248 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:19:43.276 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.355 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:43.384 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:43.589 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '2FACF508-4B1B-419D-86B4-111FA71527EB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=24891), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:43.589 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:43.589 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:43.589 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.589 4076 3051113164648 MainThread ERROR nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Traceback (most recent call last): 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(*args, **kwargs) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] admin_password, network_info, block_device_info) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(self, *args, **kwds) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] should_plug_vifs=False) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'req_state': req_state}) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] time.sleep(sleep_time) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return f(*args, **kwargs) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return self._wait_for_job(job_path) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] error_desc=err_desc) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] During handling of the above exception, another exception occurred: 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Traceback (most recent call last): 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] accel_info=accel_info) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(*args, **kwargs) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] admin_password, network_info, block_device_info) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return function(self, *args, **kwds) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] should_plug_vifs=False) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'req_state': req_state}) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] time.sleep(sleep_time) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self.force_reraise() 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] raise self.value 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return f(*args, **kwargs) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] return self._wait_for_job(job_path) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] error_desc=err_desc) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. 2021-09-10 17:19:43.589 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 2af9b131-c85d-4418-810b-00a540718354] 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Build of instance 2af9b131-c85d-4418-810b-00a540718354 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000007a' failed to start. Not enough memory in the system to start the virtual machine instance-0000007a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007a' failed to start. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Not enough memory in the system to start the virtual machine instance-0000007a with ram size 1024 megabytes. (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) 'instance-0000007a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 84E0AF18-3017-461A-ADC5-630FC0CF5DE9) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:19:43.618 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:19:43.709 4076 3051113295720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:43.709 4076 3051113295720 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:43.709 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:43.709 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:43.728 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-da3ace1f-21d1-47a6-9bef-eb455655a74e - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:43.728 4076 3051113295720 MainThread DEBUG nova.virt.hyperv.vmops [req-da3ace1f-21d1-47a6-9bef-eb455655a74e - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:43.746 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2FACF508-4B1B-419D-86B4-111FA71527EB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109681), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:43.746 4076 3051113294664 MainThread DEBUG os_win._utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 5.0303871631622314 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:43.786 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-da3ace1f-21d1-47a6-9bef-eb455655a74e - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:43.812 4076 3051113295720 MainThread INFO nova.compute.manager [req-da3ace1f-21d1-47a6-9bef-eb455655a74e - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:44.214 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '1CDD53B2-6FF0-4F30-BC58-05C7B6120B0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15648), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:44.339 4076 3051113295720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:44.339 4076 3051113295720 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:44.339 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:44.339 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:44.361 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1CDD53B2-6FF0-4F30-BC58-05C7B6120B0F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112144), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:44.361 4076 3051113295544 MainThread DEBUG os_win._utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.750662326812744 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:44.361 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-297aba2d-cab1-4597-8777-56083030a3e6 - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:44.361 4076 3051113295720 MainThread DEBUG nova.virt.hyperv.vmops [req-297aba2d-cab1-4597-8777-56083030a3e6 - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:44.401 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-297aba2d-cab1-4597-8777-56083030a3e6 - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:44.433 4076 3051113295720 MainThread INFO nova.compute.manager [req-297aba2d-cab1-4597-8777-56083030a3e6 - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:44.590 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:44.630 4076 3051113164648 MainThread INFO nova.compute.manager [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] [instance: 2af9b131-c85d-4418-810b-00a540718354] Took 1.01 seconds to deallocate network for instance. 2021-09-10 17:19:44.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:44.776 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:19:44.783 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': 'EEA92E1B-EB8F-403B-992B-BAB8B92F3C85', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16261), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:44.870 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Deleted allocations for instance 2af9b131-c85d-4418-810b-00a540718354 2021-09-10 17:19:44.903 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-ce7c9e0a-e998-4d1b-a17a-a7b837d765ba 799e4a49be0e448b9d6d54d57b7d8d1b 5cf483b0ac5349bca4c898bb3b806d21 - default default] Lock "2af9b131-c85d-4418-810b-00a540718354" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 22.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:44.918 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:44.918 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] VM Stopped (Lifecycle Event) 2021-09-10 17:19:44.918 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:44.918 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:44.918 4076 3051113294664 MainThread ERROR os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007b\' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B93110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)instance-0000007b368187DB-ABD8-4249-8059-5BADA024B931%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EEA92E1B-EB8F-403B-992B-BAB8B92F3C85', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116595), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\nNot enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)\n\n'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931)", 'ErrorSummaryDescription': "'instance-0000007b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:44.918 4076 3051113294664 MainThread ERROR nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Failed to change vm state of instance-0000007b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. 2021-09-10 17:19:44.918 4076 3051113294664 MainThread INFO nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Got request to destroy instance 2021-09-10 17:19:44.966 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-41c3b289-5a60-44ca-8574-c29fbf9c10dd - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:44.966 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-41c3b289-5a60-44ca-8574-c29fbf9c10dd - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:44.995 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-41c3b289-5a60-44ca-8574-c29fbf9c10dd - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:45.030 4076 3051113164648 MainThread INFO nova.compute.manager [req-41c3b289-5a60-44ca-8574-c29fbf9c10dd - - - - -] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:45.091 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:45.091 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:19:45.389 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': 'EC8F3620-D76B-4456-9690-D90C6B49231D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17193), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:45.513 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:45.513 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:45.513 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:45.533 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:45.533 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EC8F3620-D76B-4456-9690-D90C6B49231D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114756), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:45.533 4076 3051113295544 MainThread DEBUG os_win._utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.923350811004639 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:19:45.533 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-8dc77cca-324e-4353-ade4-b0d473cc7c5e - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:45.533 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-8dc77cca-324e-4353-ade4-b0d473cc7c5e - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:45.591 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-8dc77cca-324e-4353-ade4-b0d473cc7c5e - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:45.614 4076 3051113292728 MainThread INFO nova.compute.manager [req-8dc77cca-324e-4353-ade4-b0d473cc7c5e - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:45.984 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:45.984 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:45.984 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:19:46.542 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': 'AE05E758-276A-4202-99F6-124646ECE2F9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15312), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:46.674 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:46.674 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:46.674 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:19:46.674 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] VM Stopped (Lifecycle Event) 2021-09-10 17:19:46.688 4076 3051113295544 MainThread ERROR os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-00000079\' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE3910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000079\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)instance-000000798DCBB63C-AEE5-4C45-A106-0A494FFCBE39%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AE05E758-276A-4202-99F6-124646ECE2F9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112584), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 19, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 19, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\nNot enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)\n\n'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39)", 'ErrorSummaryDescription': "'instance-00000079' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000079.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:19:46.688 4076 3051113295544 MainThread ERROR nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Failed to change vm state of instance-00000079 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. 2021-09-10 17:19:46.688 4076 3051113295544 MainThread INFO nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Got request to destroy instance 2021-09-10 17:19:46.688 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1c76547c-ac3f-4f24-8ce8-5d50114e5d0a - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:19:46.688 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-1c76547c-ac3f-4f24-8ce8-5d50114e5d0a - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:19:46.756 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1c76547c-ac3f-4f24-8ce8-5d50114e5d0a - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:19:46.761 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:19:46.761 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:46.761 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:46.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:46.795 4076 3051113294664 MainThread DEBUG os_win.utils.compute.vmutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Successfully changed vm state of instance-0000007b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:19:46.795 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Successfully changed state of VM instance-0000007b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:19:46.795 4076 3051113164648 MainThread INFO nova.compute.manager [req-1c76547c-ac3f-4f24-8ce8-5d50114e5d0a - - - - -] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:19:46.837 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Waiting for WMI job: {'InstanceID': '7E2EE046-E9A5-4FFA-B8ED-4B08A978F7BE', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15284), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:46.973 4076 3051113294664 MainThread DEBUG os_win.utils.jobutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] WMI job succeeded: {'InstanceID': '7E2EE046-E9A5-4FFA-B8ED-4B08A978F7BE', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29824), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:19:46.973 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:46.973 4076 3051113294664 MainThread DEBUG os_win.utils.pathutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Removing directory: C:\openstack\instances\instance-0000007b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:19:46.973 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:46.973 4076 3051113294664 MainThread ERROR nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Traceback (most recent call last): 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(*args, **kwargs) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] admin_password, network_info, block_device_info) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(self, *args, **kwds) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] should_plug_vifs=False) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'req_state': req_state}) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] time.sleep(sleep_time) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return f(*args, **kwargs) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return self._wait_for_job(job_path) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] error_desc=err_desc) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During handling of the above exception, another exception occurred: 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Traceback (most recent call last): 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] yield resources 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] accel_info=accel_info) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(*args, **kwargs) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] admin_password, network_info, block_device_info) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(self, *args, **kwds) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] should_plug_vifs=False) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'req_state': req_state}) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] time.sleep(sleep_time) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return f(*args, **kwargs) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return self._wait_for_job(job_path) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] error_desc=err_desc) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:46.973 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:46.973 4076 3051113294664 MainThread INFO nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Terminating instance 2021-09-10 17:19:46.973 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:19:46.973 4076 3051113294664 MainThread INFO nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Got request to destroy instance 2021-09-10 17:19:46.997 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.vmops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:19:46.997 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:46.997 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "instance-0000007b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:46.997 4076 3051113294664 MainThread INFO nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:19:46.997 4076 3051113294664 MainThread DEBUG nova.compute.claims [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:19:46.997 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:47.158 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.hostops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:47.183 4076 3051113294664 MainThread DEBUG nova.virt.hyperv.hostops [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:47.417 4076 3051113294664 MainThread DEBUG nova.compute.provider_tree [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.scheduler.client.report [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.437s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:47.434 4076 3051113294664 MainThread ERROR nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Traceback (most recent call last): 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(*args, **kwargs) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] admin_password, network_info, block_device_info) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(self, *args, **kwds) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] should_plug_vifs=False) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'req_state': req_state}) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] time.sleep(sleep_time) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return f(*args, **kwargs) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return self._wait_for_job(job_path) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] error_desc=err_desc) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] During handling of the above exception, another exception occurred: 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Traceback (most recent call last): 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] accel_info=accel_info) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(*args, **kwargs) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] admin_password, network_info, block_device_info) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return function(self, *args, **kwds) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] should_plug_vifs=False) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'req_state': req_state}) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] time.sleep(sleep_time) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self.force_reraise() 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] raise self.value 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return f(*args, **kwargs) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] return self._wait_for_job(job_path) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] error_desc=err_desc) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. 2021-09-10 17:19:47.434 4076 3051113294664 MainThread TRACE nova.compute.manager [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.compute.utils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Build of instance 56af32cb-10b4-48fe-9adb-25b79d8d4df9 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000007b' failed to start. Not enough memory in the system to start the virtual machine instance-0000007b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007b' failed to start. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Not enough memory in the system to start the virtual machine instance-0000007b with ram size 1024 megabytes. (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) 'instance-0000007b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 368187DB-ABD8-4249-8059-5BADA024B931) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:19:47.434 4076 3051113294664 MainThread DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:19:47.995 4076 3051113294664 MainThread DEBUG nova.network.neutron [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:48.022 4076 3051113294664 MainThread INFO nova.compute.manager [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] [instance: 56af32cb-10b4-48fe-9adb-25b79d8d4df9] Took 0.59 seconds to deallocate network for instance. 2021-09-10 17:19:48.231 4076 3051113294664 MainThread INFO nova.scheduler.client.report [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Deleted allocations for instance 56af32cb-10b4-48fe-9adb-25b79d8d4df9 2021-09-10 17:19:48.277 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [req-fb19b518-0913-4b10-8353-0c74953be429 bb6b40339c39491bbe2bdee1c7c82123 fecf4120b310452eb14917a31b551102 - default default] Lock "56af32cb-10b4-48fe-9adb-25b79d8d4df9" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 24.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:48.589 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:19:48.589 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:48.589 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:48.615 4076 3051113295544 MainThread DEBUG os_win.utils.compute.vmutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Successfully changed vm state of instance-00000079 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:19:48.615 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Successfully changed state of VM instance-00000079 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:19:48.639 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '3F0F689B-22A5-4F8C-B887-C339D9EE84E5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14385), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:19:48.802 4076 3051113295544 MainThread DEBUG os_win.utils.jobutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job succeeded: {'InstanceID': '3F0F689B-22A5-4F8C-B887-C339D9EE84E5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=31108), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:19:48.802 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:48.802 4076 3051113295544 MainThread DEBUG os_win.utils.pathutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Removing directory: C:\openstack\instances\instance-00000079 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:19:48.802 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:48.809 4076 3051113295544 MainThread ERROR nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Traceback (most recent call last): 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(*args, **kwargs) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] admin_password, network_info, block_device_info) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(self, *args, **kwds) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] should_plug_vifs=False) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'req_state': req_state}) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] time.sleep(sleep_time) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return f(*args, **kwargs) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return self._wait_for_job(job_path) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] error_desc=err_desc) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During handling of the above exception, another exception occurred: 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Traceback (most recent call last): 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] yield resources 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] accel_info=accel_info) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(*args, **kwargs) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] admin_password, network_info, block_device_info) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(self, *args, **kwds) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] should_plug_vifs=False) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'req_state': req_state}) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] time.sleep(sleep_time) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return f(*args, **kwargs) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return self._wait_for_job(job_path) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] error_desc=err_desc) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:48.809 4076 3051113295544 MainThread INFO nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Terminating instance 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:19:48.809 4076 3051113295544 MainThread INFO nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Got request to destroy instance 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.vmops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-00000079" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:48.809 4076 3051113295544 MainThread INFO nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG nova.compute.claims [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:19:48.809 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:48.933 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.hostops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:48.933 4076 3051113295544 MainThread DEBUG nova.virt.hyperv.hostops [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:49.143 4076 3051113295544 MainThread DEBUG nova.compute.provider_tree [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.scheduler.client.report [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:49.160 4076 3051113295544 MainThread ERROR nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Traceback (most recent call last): 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(*args, **kwargs) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] admin_password, network_info, block_device_info) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(self, *args, **kwds) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] should_plug_vifs=False) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'req_state': req_state}) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] time.sleep(sleep_time) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return f(*args, **kwargs) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return self._wait_for_job(job_path) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] error_desc=err_desc) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] During handling of the above exception, another exception occurred: 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Traceback (most recent call last): 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] accel_info=accel_info) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(*args, **kwargs) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] admin_password, network_info, block_device_info) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return function(self, *args, **kwds) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] should_plug_vifs=False) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'req_state': req_state}) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] time.sleep(sleep_time) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self.force_reraise() 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] raise self.value 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return f(*args, **kwargs) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] return self._wait_for_job(job_path) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] error_desc=err_desc) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. 2021-09-10 17:19:49.160 4076 3051113295544 MainThread TRACE nova.compute.manager [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.compute.utils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Build of instance f78339f3-2ad5-4e9f-8fa3-f315aa3de3df was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000079' failed to start. Not enough memory in the system to start the virtual machine instance-00000079. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000079' failed to start. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Not enough memory in the system to start the virtual machine instance-00000079 with ram size 1024 megabytes. (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) 'instance-00000079' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 8DCBB63C-AEE5-4C45-A106-0A494FFCBE39) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:19:49.160 4076 3051113295544 MainThread DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:19:49.917 4076 3051113295544 MainThread DEBUG nova.network.neutron [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:49.927 4076 3051113295544 MainThread INFO nova.compute.manager [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: f78339f3-2ad5-4e9f-8fa3-f315aa3de3df] Took 0.76 seconds to deallocate network for instance. 2021-09-10 17:19:50.067 4076 3051113295544 MainThread INFO nova.scheduler.client.report [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Deleted allocations for instance f78339f3-2ad5-4e9f-8fa3-f315aa3de3df 2021-09-10 17:19:50.098 4076 3051113295544 MainThread DEBUG oslo_concurrency.lockutils [req-7a260601-43ff-4d5a-9869-5093f83b038b ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "f78339f3-2ad5-4e9f-8fa3-f315aa3de3df" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 25.265s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:51.976 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:52.114 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:52.130 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:19:52.223 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:52.245 4076 3051113295016 MainThread DEBUG nova.virt.hardware [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:19:52.245 4076 3051113295016 MainThread INFO nova.compute.claims [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Claim successful on node n-h1-808474-1 2021-09-10 17:19:52.378 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:52.394 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:52.589 4076 3051113295016 MainThread DEBUG nova.compute.provider_tree [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:52.589 4076 3051113295016 MainThread DEBUG nova.scheduler.client.report [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:52.589 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:52.589 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:19:52.698 4076 3051113295016 MainThread DEBUG nova.compute.utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:19:52.707 4076 3051113164648 GreenThread-112 DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:19:52.707 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:19:52.719 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:19:52.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:52.898 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:19:52.898 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Spawning new instance 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG nova.block_device [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:52.915 4076 3051113295016 MainThread DEBUG os_win.utils.pathutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Creating directory: C:\openstack\instances\instance-0000007c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:19:52.932 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:19:52.948 4076 3051113164648 GreenThread-112 DEBUG nova.policy [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ddf55d6b15144f37ae1d2f48a5918a49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be13a4a12834460091659ecf8dd9d41f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:19:52.948 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Resizing VHD C:\openstack\instances\instance-0000007c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:19:53.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:53.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:53.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:53.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:19:54.932 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Successfully created port: f09dcdfc-a799-46fa-bbae-4db82f933b58 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:19:56.276 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Successfully updated port: f09dcdfc-a799-46fa-bbae-4db82f933b58 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:19:56.287 4076 3051113164648 GreenThread-112 DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Acquired lock "refresh_cache-6d144805-b1e0-45a4-b3a7-71d14d6fac69" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:56.287 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:19:56.332 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Received event network-changed-f09dcdfc-a799-46fa-bbae-4db82f933b58 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:19:56.332 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Refreshing instance network info cache due to event network-changed-f09dcdfc-a799-46fa-bbae-4db82f933b58. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:19:56.426 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:19:56.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:56.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:19:56.818 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:56.818 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:56.818 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:19:56.818 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:56.818 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:57.029 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2211MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:19:57.029 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:57.174 4076 3051113164648 GreenThread-112 DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Updating instance_info_cache with network_info: [{"id": "f09dcdfc-a799-46fa-bbae-4db82f933b58", "address": "fa:16:3e:c8:17:49", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf09dcdfc-a7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:57.184 4076 3051113164648 GreenThread-112 DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Releasing lock "refresh_cache-6d144805-b1e0-45a4-b3a7-71d14d6fac69" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:57.184 4076 3051113164648 GreenThread-112 DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Instance network_info: |[{"id": "f09dcdfc-a799-46fa-bbae-4db82f933b58", "address": "fa:16:3e:c8:17:49", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf09dcdfc-a7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:19:57.184 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Preparing to wait for external event network-vif-plugged-f09dcdfc-a799-46fa-bbae-4db82f933b58 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:19:57.184 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:57.184 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:57.184 4076 3051113295016 MainThread DEBUG os_win.utils.compute.vmutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Creating VM instance-0000007c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:19:57.184 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6d144805-b1e0-45a4-b3a7-71d14d6fac69" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:19:57.184 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Refreshing network info cache for port f09dcdfc-a799-46fa-bbae-4db82f933b58 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.217 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.230 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6d144805-b1e0-45a4-b3a7-71d14d6fac69 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:19:57.230 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:19:57.230 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:19:57.249 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:19:57.301 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:19:57.516 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:19:57.532 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:19:57.532 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:19:57.532 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.500s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:19:58.074 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:19:58.411 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Updated VIF entry in instance network info cache for port f09dcdfc-a799-46fa-bbae-4db82f933b58. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:19:58.411 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Updating instance_info_cache with network_info: [{"id": "f09dcdfc-a799-46fa-bbae-4db82f933b58", "address": "fa:16:3e:c8:17:49", "network": {"id": "64cdd12b-fd00-45d8-bb80-13f83006e3d5", "bridge": null, "label": "tempest-VolumesActionsTest-1247926926-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "be13a4a12834460091659ecf8dd9d41f", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapf09dcdfc-a7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:19:58.444 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-77616a10-9330-4fe7-b5e0-33e4938a50a1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6d144805-b1e0-45a4-b3a7-71d14d6fac69" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:19:59.948 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:19:59.963 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:20:00.061 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:00.061 4076 3051113293080 MainThread DEBUG nova.virt.hardware [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:20:00.061 4076 3051113293080 MainThread INFO nova.compute.claims [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Claim successful on node n-h1-808474-1 2021-09-10 17:20:00.199 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:00.199 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:00.409 4076 3051113293080 MainThread DEBUG nova.compute.provider_tree [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:00.432 4076 3051113293080 MainThread DEBUG nova.scheduler.client.report [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:00.432 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:00.432 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:20:00.510 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:00.510 4076 3051113293080 MainThread DEBUG nova.compute.utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:20:00.521 4076 3051113164648 GreenThread-113 DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:20:00.521 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:20:00.542 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:20:00.683 4076 3051113164648 GreenThread-113 DEBUG nova.policy [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c75206b921de4d1991b46238f71d9725', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '44d93cd09cd8429aa5cdf92bb48c2d79', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:20:00.785 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:20:00.795 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Spawning new instance 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG nova.block_device [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG os_win.utils.pathutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Creating directory: C:\openstack\instances\instance-0000007d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:20:00.795 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:20:00.838 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Resizing VHD C:\openstack\instances\instance-0000007d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:20:01.474 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Successfully created port: b0c2ce01-83de-418a-a736-924bd8f391a2 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:20:02.403 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Successfully updated port: b0c2ce01-83de-418a-a736-924bd8f391a2 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:20:02.425 4076 3051113164648 GreenThread-113 DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Acquired lock "refresh_cache-54b5cdb3-8dd4-4340-8e60-87009126553e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:20:02.425 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:20:02.457 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Received event network-changed-b0c2ce01-83de-418a-a736-924bd8f391a2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:20:02.457 4076 3051113295720 MainThread DEBUG nova.compute.manager [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Refreshing instance network info cache due to event network-changed-b0c2ce01-83de-418a-a736-924bd8f391a2. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:20:02.494 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:20:02.645 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-3362850d-4d39-4185-9d98-29204ec0f382 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Received event network-vif-plugged-f09dcdfc-a799-46fa-bbae-4db82f933b58 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:20:02.645 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-3362850d-4d39-4185-9d98-29204ec0f382 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:02.645 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [req-3362850d-4d39-4185-9d98-29204ec0f382 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:02.645 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-3362850d-4d39-4185-9d98-29204ec0f382 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Processing event network-vif-plugged-f09dcdfc-a799-46fa-bbae-4db82f933b58 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:20:02.668 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Using config drive for instance 2021-09-10 17:20:02.797 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Creating config drive at C:\openstack\instances\instance-0000007c\configdrive.iso 2021-09-10 17:20:02.826 4076 3051113295016 MainThread DEBUG oslo_concurrency.processutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprbdre269 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:20:02.963 4076 3051113164648 GreenThread-113 DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Updating instance_info_cache with network_info: [{"id": "b0c2ce01-83de-418a-a736-924bd8f391a2", "address": "fa:16:3e:29:58:54", "network": {"id": "2e7fc937-9752-4db4-bb5e-b12f09e9c693", "bridge": null, "label": "tempest-network-smoke--1559154251", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "44d93cd09cd8429aa5cdf92bb48c2d79", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb0c2ce01-83", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:02.995 4076 3051113164648 GreenThread-113 DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Releasing lock "refresh_cache-54b5cdb3-8dd4-4340-8e60-87009126553e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:20:02.995 4076 3051113164648 GreenThread-113 DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Instance network_info: |[{"id": "b0c2ce01-83de-418a-a736-924bd8f391a2", "address": "fa:16:3e:29:58:54", "network": {"id": "2e7fc937-9752-4db4-bb5e-b12f09e9c693", "bridge": null, "label": "tempest-network-smoke--1559154251", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "44d93cd09cd8429aa5cdf92bb48c2d79", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb0c2ce01-83", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:20:02.995 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Preparing to wait for external event network-vif-plugged-b0c2ce01-83de-418a-a736-924bd8f391a2 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:20:02.995 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:02.995 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:02.995 4076 3051113293080 MainThread DEBUG os_win.utils.compute.vmutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Creating VM instance-0000007d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:20:02.995 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-54b5cdb3-8dd4-4340-8e60-87009126553e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:20:02.995 4076 3051113295720 MainThread DEBUG nova.network.neutron [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Refreshing network info cache for port b0c2ce01-83de-418a-a736-924bd8f391a2 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:20:03.140 4076 3051113295016 MainThread DEBUG oslo_concurrency.processutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprbdre269" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:20:03.558 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Setting boot order for instance: instance-0000007c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:20:03.605 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:20:03.636 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': 'AF705EB9-D1D6-4249-BD1F-E4065D14F858', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14865), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:03.762 4076 3051113295720 MainThread DEBUG nova.network.neutron [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Updated VIF entry in instance network info cache for port b0c2ce01-83de-418a-a736-924bd8f391a2. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:20:03.762 4076 3051113295720 MainThread DEBUG nova.network.neutron [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Updating instance_info_cache with network_info: [{"id": "b0c2ce01-83de-418a-a736-924bd8f391a2", "address": "fa:16:3e:29:58:54", "network": {"id": "2e7fc937-9752-4db4-bb5e-b12f09e9c693", "bridge": null, "label": "tempest-network-smoke--1559154251", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "44d93cd09cd8429aa5cdf92bb48c2d79", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb0c2ce01-83", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:03.777 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:03.777 4076 3051113292024 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:03.777 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:03.777 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:03.777 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AF705EB9-D1D6-4249-BD1F-E4065D14F858', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113793), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:03.777 4076 3051113295016 MainThread DEBUG os_win._utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17228317260742188 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:03.777 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [req-39491c9f-cf32-4cea-8024-fb6d822884c0 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-54b5cdb3-8dd4-4340-8e60-87009126553e" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:20:03.797 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ca5f273d-9583-4134-b089-cbee3442a01e - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:03.797 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-ca5f273d-9583-4134-b089-cbee3442a01e - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:03.826 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-ca5f273d-9583-4134-b089-cbee3442a01e - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:03.859 4076 3051113164648 MainThread INFO nova.compute.manager [req-ca5f273d-9583-4134-b089-cbee3442a01e - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:03.920 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:20:04.801 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '9EBEE62F-E2DF-4766-BFCF-D575560A2D10', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14942), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:04.939 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:04.939 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:04.939 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:04.939 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:04.951 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9EBEE62F-E2DF-4766-BFCF-D575560A2D10', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111671), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:04.951 4076 3051113295016 MainThread DEBUG os_win._utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3466696739196777 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:04.951 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1cffe97a-4dc5-4397-9948-8d7391169e28 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:04.951 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-1cffe97a-4dc5-4397-9948-8d7391169e28 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:04.999 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1cffe97a-4dc5-4397-9948-8d7391169e28 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:05.029 4076 3051113164648 MainThread INFO nova.compute.manager [req-1cffe97a-4dc5-4397-9948-8d7391169e28 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:05.987 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': 'B8EB9BD2-39A5-491E-A6C4-EDD38C4783D2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14549), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:06.107 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:06.107 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:06.125 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:06.125 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:06.125 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B8EB9BD2-39A5-491E-A6C4-EDD38C4783D2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112475), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:06.125 4076 3051113295016 MainThread DEBUG os_win._utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.520188808441162 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:06.136 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-368d934e-df51-40af-ae87-90981c40838d - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:06.136 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-368d934e-df51-40af-ae87-90981c40838d - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:06.187 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-368d934e-df51-40af-ae87-90981c40838d - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:06.187 4076 3051113164648 MainThread INFO nova.compute.manager [req-368d934e-df51-40af-ae87-90981c40838d - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:07.150 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': 'B01E34EA-B3AF-4000-9FA0-F3CDDECBC6D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15259), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:07.282 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:07.282 4076 3051113295720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:07.282 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:07.282 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:07.303 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-74ab6247-c017-4fb7-8eb6-63c5d170e6b2 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:07.303 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-74ab6247-c017-4fb7-8eb6-63c5d170e6b2 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:07.303 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B01E34EA-B3AF-4000-9FA0-F3CDDECBC6D9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110263), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:07.303 4076 3051113295016 MainThread DEBUG os_win._utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.698115587234497 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:07.343 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-74ab6247-c017-4fb7-8eb6-63c5d170e6b2 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:07.375 4076 3051113164648 MainThread INFO nova.compute.manager [req-74ab6247-c017-4fb7-8eb6-63c5d170e6b2 - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:07.648 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-c63a7618-b6cf-462a-8bd2-8af7feeaaa56 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Received event network-vif-plugged-b0c2ce01-83de-418a-a736-924bd8f391a2 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:20:07.648 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-c63a7618-b6cf-462a-8bd2-8af7feeaaa56 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:07.648 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-c63a7618-b6cf-462a-8bd2-8af7feeaaa56 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:07.648 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-c63a7618-b6cf-462a-8bd2-8af7feeaaa56 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Processing event network-vif-plugged-b0c2ce01-83de-418a-a736-924bd8f391a2 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:20:07.669 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Using config drive for instance 2021-09-10 17:20:08.222 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Creating config drive at C:\openstack\instances\instance-0000007d\configdrive.iso 2021-09-10 17:20:08.266 4076 3051113293080 MainThread DEBUG oslo_concurrency.processutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpz8i3qht4 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:20:08.446 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '6B67664C-4689-449B-AA02-0D132F005024', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14725), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:08.579 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:08.579 4076 3051113294664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:08.579 4076 3051113292728 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:08.579 4076 3051113292728 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:08.579 4076 3051113293080 MainThread DEBUG oslo_concurrency.processutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpz8i3qht4" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:20:08.604 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6B67664C-4689-449B-AA02-0D132F005024', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113419), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:08.604 4076 3051113295016 MainThread DEBUG os_win._utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.9994566440582275 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:08.604 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-c6d32b4e-2afb-424f-a3c5-ddd9a11e043f - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:08.604 4076 3051113292728 MainThread DEBUG nova.virt.hyperv.vmops [req-c6d32b4e-2afb-424f-a3c5-ddd9a11e043f - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:08.658 4076 3051113292728 MainThread DEBUG nova.compute.manager [req-c6d32b4e-2afb-424f-a3c5-ddd9a11e043f - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:08.667 4076 3051113292728 MainThread INFO nova.compute.manager [req-c6d32b4e-2afb-424f-a3c5-ddd9a11e043f - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:08.983 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Setting boot order for instance: instance-0000007d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:20:09.031 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:20:09.058 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': '0B3434E6-728F-4246-B2E2-47571EB26706', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14484), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:09.192 4076 3051113292024 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:09.192 4076 3051113292024 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:09.207 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:09.207 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:09.207 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0B3434E6-728F-4246-B2E2-47571EB26706', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115565), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:09.207 4076 3051113293080 MainThread DEBUG os_win._utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17604279518127441 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:09.207 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-ec482cab-f584-4f46-a266-8b6cf3432abe - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:09.207 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-ec482cab-f584-4f46-a266-8b6cf3432abe - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:09.267 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-ec482cab-f584-4f46-a266-8b6cf3432abe - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:09.291 4076 3051113292024 MainThread INFO nova.compute.manager [req-ec482cab-f584-4f46-a266-8b6cf3432abe - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:09.620 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '739A1D2E-7243-4531-94F1-F2B85146C128', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15682), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:09.752 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:09.752 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:09.752 4076 3051113292024 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:09.769 4076 3051113292024 MainThread INFO nova.compute.manager [-] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] VM Stopped (Lifecycle Event) 2021-09-10 17:20:09.769 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007c\' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)instance-0000007cBEA79D56-CA5E-473C-B055-BFED732E15FD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '739A1D2E-7243-4531-94F1-F2B85146C128', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110222), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\nNot enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)\n\n'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD)", 'ErrorSummaryDescription': "'instance-0000007c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:09.769 4076 3051113295016 MainThread ERROR nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Failed to change vm state of instance-0000007c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. 2021-09-10 17:20:09.769 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Got request to destroy instance 2021-09-10 17:20:09.791 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-1868b0ec-2ba5-49b8-9e81-8c9bd397263a - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:09.791 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-1868b0ec-2ba5-49b8-9e81-8c9bd397263a - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:09.838 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-1868b0ec-2ba5-49b8-9e81-8c9bd397263a - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:09.854 4076 3051113292024 MainThread INFO nova.compute.manager [req-1868b0ec-2ba5-49b8-9e81-8c9bd397263a - - - - -] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:10.219 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': 'A269CF9A-D622-4A5A-80A7-B75D262D13B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14648), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:10.373 4076 3051113292024 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:10.373 4076 3051113292024 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:10.373 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:10.373 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:10.373 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A269CF9A-D622-4A5A-80A7-B75D262D13B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113964), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 10), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:10.373 4076 3051113293080 MainThread DEBUG os_win._utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3411734104156494 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:10.392 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-48d4c1bc-25f8-48f8-8b02-229f2b2d9833 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:10.392 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-48d4c1bc-25f8-48f8-8b02-229f2b2d9833 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:10.439 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-48d4c1bc-25f8-48f8-8b02-229f2b2d9833 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:10.483 4076 3051113292024 MainThread INFO nova.compute.manager [req-48d4c1bc-25f8-48f8-8b02-229f2b2d9833 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:11.389 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': '8510AA10-97AD-45B5-99D9-1A16FCC2A72F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14449), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:11.529 4076 3051113292024 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:11.529 4076 3051113292024 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:11.529 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:11.529 4076 3051113292728 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:11.529 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8510AA10-97AD-45B5-99D9-1A16FCC2A72F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113011), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 11), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:11.529 4076 3051113293080 MainThread DEBUG os_win._utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.4973833560943604 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:11.550 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-13d2e31e-78c3-4f88-ab4a-8b9d42c50238 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:11.550 4076 3051113292024 MainThread DEBUG nova.virt.hyperv.vmops [req-13d2e31e-78c3-4f88-ab4a-8b9d42c50238 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:11.590 4076 3051113292024 MainThread DEBUG nova.compute.manager [req-13d2e31e-78c3-4f88-ab4a-8b9d42c50238 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:11.590 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:20:11.590 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:11.590 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:11.619 4076 3051113295016 MainThread DEBUG os_win.utils.compute.vmutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Successfully changed vm state of instance-0000007c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:20:11.619 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Successfully changed state of VM instance-0000007c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:20:11.630 4076 3051113292024 MainThread INFO nova.compute.manager [req-13d2e31e-78c3-4f88-ab4a-8b9d42c50238 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:11.667 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Waiting for WMI job: {'InstanceID': '98766719-FBAC-46A2-B794-367CE81FB0C4', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14805), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:11.786 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] WMI job succeeded: {'InstanceID': '98766719-FBAC-46A2-B794-367CE81FB0C4', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28811), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:20:11.786 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:11.786 4076 3051113295016 MainThread DEBUG os_win.utils.pathutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Removing directory: C:\openstack\instances\instance-0000007c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:20:11.786 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:11.786 4076 3051113295016 MainThread ERROR nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Traceback (most recent call last): 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(*args, **kwargs) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] admin_password, network_info, block_device_info) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(self, *args, **kwds) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] should_plug_vifs=False) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'req_state': req_state}) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] time.sleep(sleep_time) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return f(*args, **kwargs) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return self._wait_for_job(job_path) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] error_desc=err_desc) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During handling of the above exception, another exception occurred: 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Traceback (most recent call last): 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] yield resources 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] accel_info=accel_info) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(*args, **kwargs) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] admin_password, network_info, block_device_info) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(self, *args, **kwds) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] should_plug_vifs=False) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'req_state': req_state}) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] time.sleep(sleep_time) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return f(*args, **kwargs) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return self._wait_for_job(job_path) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] error_desc=err_desc) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:11.786 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:11.786 4076 3051113295016 MainThread INFO nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Terminating instance 2021-09-10 17:20:11.786 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:20:11.800 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Got request to destroy instance 2021-09-10 17:20:11.800 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:20:11.822 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:11.822 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "instance-0000007c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:11.822 4076 3051113295016 MainThread INFO nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:20:11.822 4076 3051113295016 MainThread DEBUG nova.compute.claims [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:20:11.822 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:11.918 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:11.918 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:12.141 4076 3051113295016 MainThread DEBUG nova.compute.provider_tree [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.scheduler.client.report [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:12.155 4076 3051113295016 MainThread ERROR nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Traceback (most recent call last): 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(*args, **kwargs) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] admin_password, network_info, block_device_info) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(self, *args, **kwds) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] should_plug_vifs=False) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'req_state': req_state}) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] time.sleep(sleep_time) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return f(*args, **kwargs) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return self._wait_for_job(job_path) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] error_desc=err_desc) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] During handling of the above exception, another exception occurred: 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Traceback (most recent call last): 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] accel_info=accel_info) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(*args, **kwargs) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] admin_password, network_info, block_device_info) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return function(self, *args, **kwds) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] should_plug_vifs=False) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'req_state': req_state}) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] time.sleep(sleep_time) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self.force_reraise() 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] raise self.value 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return f(*args, **kwargs) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] return self._wait_for_job(job_path) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] error_desc=err_desc) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. 2021-09-10 17:20:12.155 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.compute.utils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Build of instance 6d144805-b1e0-45a4-b3a7-71d14d6fac69 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000007c' failed to start. Not enough memory in the system to start the virtual machine instance-0000007c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007c' failed to start. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Not enough memory in the system to start the virtual machine instance-0000007c with ram size 1024 megabytes. (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) 'instance-0000007c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID BEA79D56-CA5E-473C-B055-BFED732E15FD) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:20:12.155 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:20:12.557 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': '3F8E044C-2F28-4C33-BE82-612967758108', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15204), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:12.688 4076 3051113293256 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:12.688 4076 3051113293256 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:12.688 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:12.688 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:12.688 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3F8E044C-2F28-4C33-BE82-612967758108', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110615), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:12.688 4076 3051113293080 MainThread DEBUG os_win._utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6567323207855225 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:12.688 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-3e7c7a3c-1b47-441b-9b50-31a5dcf0faa2 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:12.713 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-3e7c7a3c-1b47-441b-9b50-31a5dcf0faa2 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:12.730 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-3e7c7a3c-1b47-441b-9b50-31a5dcf0faa2 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:12.771 4076 3051113293256 MainThread INFO nova.compute.manager [req-3e7c7a3c-1b47-441b-9b50-31a5dcf0faa2 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:12.916 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:12.943 4076 3051113295016 MainThread INFO nova.compute.manager [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] [instance: 6d144805-b1e0-45a4-b3a7-71d14d6fac69] Took 0.80 seconds to deallocate network for instance. 2021-09-10 17:20:13.156 4076 3051113295016 MainThread INFO nova.scheduler.client.report [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Deleted allocations for instance 6d144805-b1e0-45a4-b3a7-71d14d6fac69 2021-09-10 17:20:13.197 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-cb96e84e-46ef-447e-9447-b4d746b4170e ddf55d6b15144f37ae1d2f48a5918a49 be13a4a12834460091659ecf8dd9d41f - default default] Lock "6d144805-b1e0-45a4-b3a7-71d14d6fac69" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 21.078s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:13.689 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': '5BD378D2-9425-49AF-8AB5-DAC5D934E57F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14202), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:13.835 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:13.835 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:13.845 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:13.845 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:13.870 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1680f815-df51-493d-8d5e-2f9495138c35 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:13.870 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-1680f815-df51-493d-8d5e-2f9495138c35 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:13.919 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-1680f815-df51-493d-8d5e-2f9495138c35 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:13.950 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5BD378D2-9425-49AF-8AB5-DAC5D934E57F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111284), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:13.950 4076 3051113293080 MainThread DEBUG os_win._utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.918214321136475 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:13.950 4076 3051113164648 MainThread INFO nova.compute.manager [req-1680f815-df51-493d-8d5e-2f9495138c35 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:14.974 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': 'D83EDF53-95B1-4D2B-BCA9-4DF52DFC1167', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14865), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:15.091 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:15.091 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] VM Stopped (Lifecycle Event) 2021-09-10 17:20:15.091 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:15.091 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:15.135 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007d\' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)instance-0000007dE03DF28E-7A76-4003-9004-B112DBE0ACB8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D83EDF53-95B1-4D2B-BCA9-4DF52DFC1167', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111389), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\nNot enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)\n\n'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8)", 'ErrorSummaryDescription': "'instance-0000007d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:15.135 4076 3051113293080 MainThread ERROR nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Failed to change vm state of instance-0000007d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. 2021-09-10 17:20:15.135 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Got request to destroy instance 2021-09-10 17:20:15.135 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-99a7c270-7a57-41c8-99e6-776f99c1aa75 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:15.135 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-99a7c270-7a57-41c8-99e6-776f99c1aa75 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:15.197 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-99a7c270-7a57-41c8-99e6-776f99c1aa75 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:15.229 4076 3051113164648 MainThread INFO nova.compute.manager [req-99a7c270-7a57-41c8-99e6-776f99c1aa75 - - - - -] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:16.932 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:20:16.932 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:16.932 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:16.963 4076 3051113293080 MainThread DEBUG os_win.utils.compute.vmutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Successfully changed vm state of instance-0000007d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:20:16.963 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Successfully changed state of VM instance-0000007d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:20:17.010 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Waiting for WMI job: {'InstanceID': '8AEDEEB7-8F34-439F-97F7-875C7A23C805', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16256), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:17.138 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] WMI job succeeded: {'InstanceID': '8AEDEEB7-8F34-439F-97F7-875C7A23C805', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=46823), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:20:17.138 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:17.138 4076 3051113293080 MainThread DEBUG os_win.utils.pathutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Removing directory: C:\openstack\instances\instance-0000007d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:20:17.138 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:17.138 4076 3051113293080 MainThread ERROR nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Traceback (most recent call last): 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(*args, **kwargs) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] admin_password, network_info, block_device_info) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(self, *args, **kwds) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] should_plug_vifs=False) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'req_state': req_state}) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] time.sleep(sleep_time) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return f(*args, **kwargs) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return self._wait_for_job(job_path) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] error_desc=err_desc) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During handling of the above exception, another exception occurred: 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Traceback (most recent call last): 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] yield resources 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] accel_info=accel_info) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(*args, **kwargs) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] admin_password, network_info, block_device_info) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(self, *args, **kwds) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] should_plug_vifs=False) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'req_state': req_state}) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] time.sleep(sleep_time) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return f(*args, **kwargs) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return self._wait_for_job(job_path) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] error_desc=err_desc) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.138 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.138 4076 3051113293080 MainThread INFO nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Terminating instance 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:20:17.153 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Got request to destroy instance 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "instance-0000007d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:17.153 4076 3051113293080 MainThread INFO nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG nova.compute.claims [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:20:17.153 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:17.262 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:17.277 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:17.542 4076 3051113293080 MainThread DEBUG nova.compute.provider_tree [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:17.557 4076 3051113293080 MainThread DEBUG nova.scheduler.client.report [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:17.557 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:17.557 4076 3051113293080 MainThread ERROR nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Traceback (most recent call last): 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(*args, **kwargs) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] admin_password, network_info, block_device_info) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(self, *args, **kwds) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] should_plug_vifs=False) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'req_state': req_state}) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] time.sleep(sleep_time) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return f(*args, **kwargs) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return self._wait_for_job(job_path) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] error_desc=err_desc) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] During handling of the above exception, another exception occurred: 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Traceback (most recent call last): 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] accel_info=accel_info) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(*args, **kwargs) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] admin_password, network_info, block_device_info) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return function(self, *args, **kwds) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] should_plug_vifs=False) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'req_state': req_state}) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] time.sleep(sleep_time) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self.force_reraise() 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] raise self.value 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return f(*args, **kwargs) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] return self._wait_for_job(job_path) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] error_desc=err_desc) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. 2021-09-10 17:20:17.557 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] 2021-09-10 17:20:17.557 4076 3051113293080 MainThread DEBUG nova.compute.utils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:20:17.557 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Build of instance 54b5cdb3-8dd4-4340-8e60-87009126553e was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000007d' failed to start. Not enough memory in the system to start the virtual machine instance-0000007d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007d' failed to start. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Not enough memory in the system to start the virtual machine instance-0000007d with ram size 1024 megabytes. (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) 'instance-0000007d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID E03DF28E-7A76-4003-9004-B112DBE0ACB8) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:20:17.557 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:20:17.580 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:20:17.580 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:20:17.580 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:20:18.425 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:18.452 4076 3051113293080 MainThread INFO nova.compute.manager [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] [instance: 54b5cdb3-8dd4-4340-8e60-87009126553e] Took 0.88 seconds to deallocate network for instance. 2021-09-10 17:20:18.646 4076 3051113293080 MainThread INFO nova.scheduler.client.report [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Deleted allocations for instance 54b5cdb3-8dd4-4340-8e60-87009126553e 2021-09-10 17:20:18.697 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-48482a72-411f-4974-8347-26d29d68347a c75206b921de4d1991b46238f71d9725 44d93cd09cd8429aa5cdf92bb48c2d79 - default default] Lock "54b5cdb3-8dd4-4340-8e60-87009126553e" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.750s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:19.462 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:19.470 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:20:19.615 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:19.615 4076 3051113164648 MainThread DEBUG nova.virt.hardware [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:20:19.615 4076 3051113164648 MainThread INFO nova.compute.claims [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Claim successful on node n-h1-808474-1 2021-09-10 17:20:19.760 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:19.775 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:19.979 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:19.979 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:19.979 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:19.979 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:20:20.057 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:20:20.057 4076 3051113293080 GreenThread-114 DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:20:20.057 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:20:20.098 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:20:20.242 4076 3051113293080 GreenThread-114 DEBUG nova.policy [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '47eebdda1ffa45eaa1d396ecfcf5780b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd2e7cb91cb99496aae913ddb707af6c4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:20:20.304 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:20:20.304 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Spawning new instance 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG nova.block_device [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Creating directory: C:\openstack\instances\instance-0000007e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:20:20.326 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:20:20.354 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Resizing VHD C:\openstack\instances\instance-0000007e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:20:21.330 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Successfully created port: a6c93850-2a25-4141-b701-e44872be9bfd _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:20:22.541 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Successfully updated port: a6c93850-2a25-4141-b701-e44872be9bfd _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:20:22.562 4076 3051113293080 GreenThread-114 DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Acquired lock "refresh_cache-96ec5341-0317-49a0-869e-4ccbd1788cf1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:20:22.562 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:20:22.624 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Received event network-changed-a6c93850-2a25-4141-b701-e44872be9bfd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:20:22.624 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Refreshing instance network info cache due to event network-changed-a6c93850-2a25-4141-b701-e44872be9bfd. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:20:22.692 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:20:23.088 4076 3051113293080 GreenThread-114 DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Updating instance_info_cache with network_info: [{"id": "a6c93850-2a25-4141-b701-e44872be9bfd", "address": "fa:16:3e:1d:6f:a9", "network": {"id": "1e50d601-5a38-4221-ba03-d236b39fa882", "bridge": null, "label": "tempest-TestServerAdvancedOps-1310542228-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "d2e7cb91cb99496aae913ddb707af6c4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa6c93850-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:23.103 4076 3051113293080 GreenThread-114 DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Releasing lock "refresh_cache-96ec5341-0317-49a0-869e-4ccbd1788cf1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:20:23.103 4076 3051113293080 GreenThread-114 DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Instance network_info: |[{"id": "a6c93850-2a25-4141-b701-e44872be9bfd", "address": "fa:16:3e:1d:6f:a9", "network": {"id": "1e50d601-5a38-4221-ba03-d236b39fa882", "bridge": null, "label": "tempest-TestServerAdvancedOps-1310542228-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "d2e7cb91cb99496aae913ddb707af6c4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa6c93850-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:20:23.103 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Preparing to wait for external event network-vif-plugged-a6c93850-2a25-4141-b701-e44872be9bfd prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:20:23.103 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:23.103 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:23.103 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Creating VM instance-0000007e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:20:23.119 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-96ec5341-0317-49a0-869e-4ccbd1788cf1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:20:23.119 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Refreshing network info cache for port a6c93850-2a25-4141-b701-e44872be9bfd _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:20:23.994 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:20:24.166 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Updated VIF entry in instance network info cache for port a6c93850-2a25-4141-b701-e44872be9bfd. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:20:24.166 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Updating instance_info_cache with network_info: [{"id": "a6c93850-2a25-4141-b701-e44872be9bfd", "address": "fa:16:3e:1d:6f:a9", "network": {"id": "1e50d601-5a38-4221-ba03-d236b39fa882", "bridge": null, "label": "tempest-TestServerAdvancedOps-1310542228-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "d2e7cb91cb99496aae913ddb707af6c4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapa6c93850-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:24.194 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-4c9c29d6-eaf5-47c1-a2b3-521411e6d8b7 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-96ec5341-0317-49a0-869e-4ccbd1788cf1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:20:27.759 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7dc537b4-0a41-4e46-8be0-509890d79764 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Received event network-vif-plugged-a6c93850-2a25-4141-b701-e44872be9bfd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:20:27.775 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7dc537b4-0a41-4e46-8be0-509890d79764 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:27.775 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7dc537b4-0a41-4e46-8be0-509890d79764 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:27.775 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7dc537b4-0a41-4e46-8be0-509890d79764 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Processing event network-vif-plugged-a6c93850-2a25-4141-b701-e44872be9bfd _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:20:27.806 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Using config drive for instance 2021-09-10 17:20:27.916 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Creating config drive at C:\openstack\instances\instance-0000007e\configdrive.iso 2021-09-10 17:20:27.934 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpnmp1xwi2 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:20:28.234 4076 3051113164648 MainThread DEBUG oslo_concurrency.processutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpnmp1xwi2" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:20:28.641 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Setting boot order for instance: instance-0000007e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:20:28.684 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:20:28.731 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': '942735A2-AC66-4524-8FDD-C787FCD9835E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14452), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:28.860 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:28.860 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:28.875 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:28.875 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:28.885 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '942735A2-AC66-4524-8FDD-C787FCD9835E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112642), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:28.885 4076 3051113164648 MainThread DEBUG os_win._utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18482446670532227 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:28.885 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9576af40-6257-44b1-a3e6-bc7ea6e4982e - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:28.885 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-9576af40-6257-44b1-a3e6-bc7ea6e4982e - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:28.936 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9576af40-6257-44b1-a3e6-bc7ea6e4982e - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:28.962 4076 3051113293080 MainThread INFO nova.compute.manager [req-9576af40-6257-44b1-a3e6-bc7ea6e4982e - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:29.916 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': 'CEC67692-56D3-427B-9D52-0F1629BBBDAF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15030), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:30.050 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:30.050 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:30.050 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:30.050 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:30.050 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CEC67692-56D3-427B-9D52-0F1629BBBDAF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111216), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:30.050 4076 3051113164648 MainThread DEBUG os_win._utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3497633934020996 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:30.065 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-ddd6970a-bae3-4b55-8b4c-c489798a9626 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:30.072 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-ddd6970a-bae3-4b55-8b4c-c489798a9626 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:30.115 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-ddd6970a-bae3-4b55-8b4c-c489798a9626 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:30.140 4076 3051113293080 MainThread INFO nova.compute.manager [req-ddd6970a-bae3-4b55-8b4c-c489798a9626 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:31.085 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': 'BF05A9E3-5F83-4A18-89FE-DEF52C9A1D07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15308), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:31.215 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:31.215 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:31.215 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:31.215 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:31.240 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-d632f3b0-e3fc-4e39-9295-4f4b02ef7287 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:31.240 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-d632f3b0-e3fc-4e39-9295-4f4b02ef7287 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:31.240 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BF05A9E3-5F83-4A18-89FE-DEF52C9A1D07', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109275), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:31.240 4076 3051113164648 MainThread DEBUG os_win._utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.539630651473999 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:31.282 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-d632f3b0-e3fc-4e39-9295-4f4b02ef7287 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:31.309 4076 3051113293080 MainThread INFO nova.compute.manager [req-d632f3b0-e3fc-4e39-9295-4f4b02ef7287 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:32.248 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': 'FE819C7E-032B-4B61-8080-96A506FAC3B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14608), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:32.397 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:32.397 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:32.397 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:32.397 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:32.423 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9219f589-b1a4-4896-8660-5b9507f9edf8 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:32.423 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-9219f589-b1a4-4896-8660-5b9507f9edf8 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:32.456 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-9219f589-b1a4-4896-8660-5b9507f9edf8 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:32.494 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FE819C7E-032B-4B61-8080-96A506FAC3B1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110882), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:32.494 4076 3051113164648 MainThread DEBUG os_win._utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7934539318084717 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:32.494 4076 3051113293080 MainThread INFO nova.compute.manager [req-9219f589-b1a4-4896-8660-5b9507f9edf8 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:33.518 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': 'BCA74264-BBCE-46E2-8D5F-B93E145C68E4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14103), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:33.659 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BCA74264-BBCE-46E2-8D5F-B93E145C68E4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111214), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:33.666 4076 3051113164648 MainThread DEBUG os_win._utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.96596622467041 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:20:33.666 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:33.666 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:33.666 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:33.666 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:33.697 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-6eac32f3-8ca8-47ff-bee3-c737023205db - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:33.697 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-6eac32f3-8ca8-47ff-bee3-c737023205db - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:33.728 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-6eac32f3-8ca8-47ff-bee3-c737023205db - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:33.764 4076 3051113293080 MainThread INFO nova.compute.manager [req-6eac32f3-8ca8-47ff-bee3-c737023205db - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:34.681 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': 'E8B05478-79EE-473A-A030-33D84BB6761F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15146), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:34.825 4076 3051113293080 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:20:34.825 4076 3051113293080 MainThread INFO nova.compute.manager [-] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] VM Stopped (Lifecycle Event) 2021-09-10 17:20:34.825 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:34.825 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:34.838 4076 3051113164648 MainThread ERROR os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007e\' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF598110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)instance-0000007e942D8536-E825-4A46-91F0-BBB91DAF5981%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E8B05478-79EE-473A-A030-33D84BB6761F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110661), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 20, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 20, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\nNot enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)\n\n'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981)", 'ErrorSummaryDescription': "'instance-0000007e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:20:34.853 4076 3051113164648 MainThread ERROR nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Failed to change vm state of instance-0000007e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. 2021-09-10 17:20:34.853 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Got request to destroy instance 2021-09-10 17:20:34.853 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-8ace06c3-f2d6-458f-8804-13cfcd8b01f9 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:20:34.853 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-8ace06c3-f2d6-458f-8804-13cfcd8b01f9 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:20:34.886 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-8ace06c3-f2d6-458f-8804-13cfcd8b01f9 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:20:34.916 4076 3051113293080 MainThread INFO nova.compute.manager [req-8ace06c3-f2d6-458f-8804-13cfcd8b01f9 - - - - -] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:20:36.632 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:20:36.632 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:36.632 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:36.653 4076 3051113164648 MainThread DEBUG os_win.utils.compute.vmutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Successfully changed vm state of instance-0000007e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:20:36.653 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Successfully changed state of VM instance-0000007e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:20:36.692 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Waiting for WMI job: {'InstanceID': '70E82DD4-FB46-4319-8D4A-4B1D4D485F52', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14005), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:20:36.802 4076 3051113164648 MainThread DEBUG os_win.utils.jobutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] WMI job succeeded: {'InstanceID': '70E82DD4-FB46-4319-8D4A-4B1D4D485F52', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30111), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:20:36.802 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:36.802 4076 3051113164648 MainThread DEBUG os_win.utils.pathutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Removing directory: C:\openstack\instances\instance-0000007e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:20:36.822 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:36.822 4076 3051113164648 MainThread ERROR nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Traceback (most recent call last): 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(*args, **kwargs) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] admin_password, network_info, block_device_info) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(self, *args, **kwds) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] should_plug_vifs=False) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'req_state': req_state}) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] time.sleep(sleep_time) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return f(*args, **kwargs) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return self._wait_for_job(job_path) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] error_desc=err_desc) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During handling of the above exception, another exception occurred: 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Traceback (most recent call last): 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] yield resources 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] accel_info=accel_info) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(*args, **kwargs) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] admin_password, network_info, block_device_info) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(self, *args, **kwds) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] should_plug_vifs=False) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'req_state': req_state}) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] time.sleep(sleep_time) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return f(*args, **kwargs) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return self._wait_for_job(job_path) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] error_desc=err_desc) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:36.822 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:36.822 4076 3051113164648 MainThread INFO nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Terminating instance 2021-09-10 17:20:36.822 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:20:36.829 4076 3051113164648 MainThread INFO nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Got request to destroy instance 2021-09-10 17:20:36.829 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:20:36.829 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:36.829 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "instance-0000007e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:36.829 4076 3051113164648 MainThread INFO nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:20:36.853 4076 3051113164648 MainThread DEBUG nova.compute.claims [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:20:36.853 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:36.943 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:36.956 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.hostops [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:37.151 4076 3051113164648 MainThread DEBUG nova.compute.provider_tree [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:37.151 4076 3051113164648 MainThread DEBUG nova.scheduler.client.report [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:37.151 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:37.151 4076 3051113164648 MainThread ERROR nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Traceback (most recent call last): 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(*args, **kwargs) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] admin_password, network_info, block_device_info) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(self, *args, **kwds) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] should_plug_vifs=False) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'req_state': req_state}) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] time.sleep(sleep_time) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return f(*args, **kwargs) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return self._wait_for_job(job_path) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] error_desc=err_desc) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] During handling of the above exception, another exception occurred: 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Traceback (most recent call last): 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] accel_info=accel_info) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(*args, **kwargs) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] admin_password, network_info, block_device_info) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return function(self, *args, **kwds) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] should_plug_vifs=False) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'req_state': req_state}) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] time.sleep(sleep_time) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self.force_reraise() 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] raise self.value 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return f(*args, **kwargs) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] return self._wait_for_job(job_path) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] error_desc=err_desc) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. 2021-09-10 17:20:37.151 4076 3051113164648 MainThread TRACE nova.compute.manager [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.compute.utils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Build of instance 96ec5341-0317-49a0-869e-4ccbd1788cf1 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000007e' failed to start. Not enough memory in the system to start the virtual machine instance-0000007e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007e' failed to start. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Not enough memory in the system to start the virtual machine instance-0000007e with ram size 1024 megabytes. (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) 'instance-0000007e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 942D8536-E825-4A46-91F0-BBB91DAF5981) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:20:37.181 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:20:37.697 4076 3051113164648 MainThread DEBUG nova.network.neutron [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:37.723 4076 3051113164648 MainThread INFO nova.compute.manager [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] [instance: 96ec5341-0317-49a0-869e-4ccbd1788cf1] Took 0.55 seconds to deallocate network for instance. 2021-09-10 17:20:37.965 4076 3051113164648 MainThread INFO nova.scheduler.client.report [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Deleted allocations for instance 96ec5341-0317-49a0-869e-4ccbd1788cf1 2021-09-10 17:20:38.025 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-a0bd4937-3ac3-42d3-91a3-72469b776398 47eebdda1ffa45eaa1d396ecfcf5780b d2e7cb91cb99496aae913ddb707af6c4 - default default] Lock "96ec5341-0317-49a0-869e-4ccbd1788cf1" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.562s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:45.806 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:45.806 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:20:45.965 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:20:45.965 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:20:46.904 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:20:46.932 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:20:46.932 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:20:46.932 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:53.777 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:53.777 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:53.833 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:53.833 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:53.833 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:20:55.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:56.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:56.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:20:56.793 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:56.793 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:20:56.793 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:20:56.793 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:56.793 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:57.009 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2209MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:20:57.009 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:20:57.269 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:20:57.290 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:20:57.325 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:20:57.348 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:20:57.547 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:20:57.575 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:20:57.575 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:20:57.575 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.563s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:02.571 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:21.362 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:21.399 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:21:21.565 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:21.565 4076 3051113294840 MainThread DEBUG nova.virt.hardware [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:21:21.565 4076 3051113294840 MainThread INFO nova.compute.claims [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Claim successful on node n-h1-808474-1 2021-09-10 17:21:21.920 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:21:21.932 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:21:22.122 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:21:22.144 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:21:22.144 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.578s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:22.144 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:21:22.278 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:21:22.278 4076 3051113164648 GreenThread-115 DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:21:22.278 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:21:22.315 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:21:22.700 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:21:22.700 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Spawning new instance 2021-09-10 17:21:22.727 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:22.727 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:22.727 4076 3051113294840 MainThread DEBUG nova.block_device [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:21:22.733 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:22.733 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:22.733 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Creating directory: C:\openstack\instances\instance-0000007f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:21:22.733 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000007f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:21:22.759 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Resizing VHD C:\openstack\instances\instance-0000007f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:21:23.720 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Successfully created port: baceb3f5-6ef6-4b17-b704-b12aea671602 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:21:24.782 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Successfully updated port: baceb3f5-6ef6-4b17-b704-b12aea671602 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:21:24.821 4076 3051113164648 GreenThread-115 DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Acquired lock "refresh_cache-4af2d18a-b831-4268-8b2d-64047ec5e874" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:21:24.821 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:21:24.947 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:21:25.098 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Received event network-changed-baceb3f5-6ef6-4b17-b704-b12aea671602 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:21:25.098 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Refreshing instance network info cache due to event network-changed-baceb3f5-6ef6-4b17-b704-b12aea671602. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:21:25.591 4076 3051113164648 GreenThread-115 DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Updating instance_info_cache with network_info: [{"id": "baceb3f5-6ef6-4b17-b704-b12aea671602", "address": "fa:16:3e:2e:d7:18", "network": {"id": "304f3a87-19cc-4862-90aa-d0722f1018fc", "bridge": null, "label": "tempest-TestServerMultinode-1190428048-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "989765f44a794ecd9cb778fb40d971d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbaceb3f5-6e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:21:25.633 4076 3051113164648 GreenThread-115 DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Releasing lock "refresh_cache-4af2d18a-b831-4268-8b2d-64047ec5e874" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:21:25.633 4076 3051113164648 GreenThread-115 DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Instance network_info: |[{"id": "baceb3f5-6ef6-4b17-b704-b12aea671602", "address": "fa:16:3e:2e:d7:18", "network": {"id": "304f3a87-19cc-4862-90aa-d0722f1018fc", "bridge": null, "label": "tempest-TestServerMultinode-1190428048-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "989765f44a794ecd9cb778fb40d971d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbaceb3f5-6e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:21:25.633 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-4af2d18a-b831-4268-8b2d-64047ec5e874" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:21:25.633 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Refreshing network info cache for port baceb3f5-6ef6-4b17-b704-b12aea671602 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:21:25.633 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Preparing to wait for external event network-vif-plugged-baceb3f5-6ef6-4b17-b704-b12aea671602 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:21:25.633 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:25.633 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:25.633 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Creating VM instance-0000007f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:21:26.463 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Updated VIF entry in instance network info cache for port baceb3f5-6ef6-4b17-b704-b12aea671602. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:21:26.463 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Updating instance_info_cache with network_info: [{"id": "baceb3f5-6ef6-4b17-b704-b12aea671602", "address": "fa:16:3e:2e:d7:18", "network": {"id": "304f3a87-19cc-4862-90aa-d0722f1018fc", "bridge": null, "label": "tempest-TestServerMultinode-1190428048-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "989765f44a794ecd9cb778fb40d971d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbaceb3f5-6e", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:21:26.485 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-9d2d4a5c-561e-4851-b40f-dd8c9d3e0628 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-4af2d18a-b831-4268-8b2d-64047ec5e874" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:21:26.512 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:21:32.223 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-f8fcf685-a07f-4c50-82b8-025ea1374e1c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Received event network-vif-plugged-baceb3f5-6ef6-4b17-b704-b12aea671602 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:21:32.223 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-f8fcf685-a07f-4c50-82b8-025ea1374e1c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:32.223 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-f8fcf685-a07f-4c50-82b8-025ea1374e1c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:32.223 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-f8fcf685-a07f-4c50-82b8-025ea1374e1c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Processing event network-vif-plugged-baceb3f5-6ef6-4b17-b704-b12aea671602 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:21:32.247 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Using config drive for instance 2021-09-10 17:21:32.426 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Creating config drive at C:\openstack\instances\instance-0000007f\configdrive.iso 2021-09-10 17:21:32.461 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpuoh4m22r execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:21:32.785 4076 3051113294840 MainThread DEBUG oslo_concurrency.processutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000007f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpuoh4m22r" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:21:33.183 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Setting boot order for instance: instance-0000007f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:21:33.237 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:21:33.265 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '6F37884A-C1A2-458C-9C9D-09B24DE72EFC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14247), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:33.387 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6F37884A-C1A2-458C-9C9D-09B24DE72EFC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110902), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:33.387 4076 3051113294840 MainThread DEBUG os_win._utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.15039277076721191 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:21:33.414 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:33.414 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:33.414 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:33.414 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:33.440 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-3cf26dbc-a476-43e1-b4f3-c580c074ee14 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:33.440 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-3cf26dbc-a476-43e1-b4f3-c580c074ee14 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:33.478 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-3cf26dbc-a476-43e1-b4f3-c580c074ee14 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:33.514 4076 3051113164648 MainThread INFO nova.compute.manager [req-3cf26dbc-a476-43e1-b4f3-c580c074ee14 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:34.414 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '546837A6-0BDF-406E-8AB6-1D9B413EBB8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14679), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:34.539 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:34.539 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:34.549 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:34.549 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:34.574 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-23bca77a-b4dc-4c09-b1a5-f3a028b471be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:34.574 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-23bca77a-b4dc-4c09-b1a5-f3a028b471be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:34.574 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '546837A6-0BDF-406E-8AB6-1D9B413EBB8B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109988), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:34.574 4076 3051113294840 MainThread DEBUG os_win._utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3374571800231934 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:21:34.623 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-23bca77a-b4dc-4c09-b1a5-f3a028b471be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:34.623 4076 3051113164648 MainThread INFO nova.compute.manager [req-23bca77a-b4dc-4c09-b1a5-f3a028b471be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:35.609 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '8D64976D-9E74-413D-8ACC-EC191CCAB01F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14665), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:35.730 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:35.730 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:35.730 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:35.730 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:35.752 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8D64976D-9E74-413D-8ACC-EC191CCAB01F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109021), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:35.752 4076 3051113294840 MainThread DEBUG os_win._utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.515331983566284 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:21:35.752 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-4cb837d9-bcca-4399-9a85-cf9b949586be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:35.752 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-4cb837d9-bcca-4399-9a85-cf9b949586be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:35.813 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-4cb837d9-bcca-4399-9a85-cf9b949586be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:35.832 4076 3051113164648 MainThread INFO nova.compute.manager [req-4cb837d9-bcca-4399-9a85-cf9b949586be - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:36.774 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': 'A5DB9F6E-3A6A-4F5C-8979-7E0C82DCD713', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18857), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:36.913 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:36.913 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:36.913 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:36.913 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:36.913 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A5DB9F6E-3A6A-4F5C-8979-7E0C82DCD713', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109579), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:36.913 4076 3051113294840 MainThread DEBUG os_win._utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.675992250442505 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:21:36.926 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-0fdf250c-6e52-40ed-ac23-c53fc39da953 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:36.926 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-0fdf250c-6e52-40ed-ac23-c53fc39da953 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:36.977 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-0fdf250c-6e52-40ed-ac23-c53fc39da953 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:37.004 4076 3051113164648 MainThread INFO nova.compute.manager [req-0fdf250c-6e52-40ed-ac23-c53fc39da953 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:37.967 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '25FE318D-C055-4E80-86D1-DE8262C91138', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15232), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:38.101 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:38.101 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:38.101 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:38.101 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:38.111 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '25FE318D-C055-4E80-86D1-DE8262C91138', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113442), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:38.111 4076 3051113294840 MainThread DEBUG os_win._utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.874425172805786 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:21:38.129 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-50690382-755a-436c-97ad-a50839c67625 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:38.129 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-50690382-755a-436c-97ad-a50839c67625 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:38.149 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-50690382-755a-436c-97ad-a50839c67625 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:38.183 4076 3051113164648 MainThread INFO nova.compute.manager [req-50690382-755a-436c-97ad-a50839c67625 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:39.136 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '737585E6-6F7D-4202-B780-52F1C4C9BC95', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15202), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:39.279 4076 3051113294840 MainThread ERROR os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000007f\' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62412030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB62410243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000007f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)instance-0000007f7E726BB3-5FA2-4188-BD9A-8681613DB624%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '737585E6-6F7D-4202-B780-52F1C4C9BC95', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111429), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 21, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 21, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\nNot enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)\n\n'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624)", 'ErrorSummaryDescription': "'instance-0000007f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000007f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:21:39.279 4076 3051113294840 MainThread ERROR nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Failed to change vm state of instance-0000007f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. 2021-09-10 17:21:39.279 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Got request to destroy instance 2021-09-10 17:21:39.279 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:21:39.279 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] VM Stopped (Lifecycle Event) 2021-09-10 17:21:39.279 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:39.279 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:39.306 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-23c2864d-7705-4e7a-8262-7fb81d22e2b6 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:21:39.306 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-23c2864d-7705-4e7a-8262-7fb81d22e2b6 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:21:39.348 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-23c2864d-7705-4e7a-8262-7fb81d22e2b6 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:21:39.364 4076 3051113164648 MainThread INFO nova.compute.manager [req-23c2864d-7705-4e7a-8262-7fb81d22e2b6 - - - - -] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:21:41.079 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:21:41.079 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:41.079 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:41.079 4076 3051113294840 MainThread DEBUG os_win.utils.compute.vmutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Successfully changed vm state of instance-0000007f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:21:41.079 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Successfully changed state of VM instance-0000007f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:21:41.128 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Waiting for WMI job: {'InstanceID': '6E72D753-1366-44D1-80D8-50E7333CE0A0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14389), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:21:41.244 4076 3051113294840 MainThread DEBUG os_win.utils.jobutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] WMI job succeeded: {'InstanceID': '6E72D753-1366-44D1-80D8-50E7333CE0A0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30633), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:21:41.244 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:41.244 4076 3051113294840 MainThread DEBUG os_win.utils.pathutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Removing directory: C:\openstack\instances\instance-0000007f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:21:41.264 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:41.264 4076 3051113294840 MainThread ERROR nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Traceback (most recent call last): 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(*args, **kwargs) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] admin_password, network_info, block_device_info) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(self, *args, **kwds) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] should_plug_vifs=False) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'req_state': req_state}) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] time.sleep(sleep_time) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return f(*args, **kwargs) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return self._wait_for_job(job_path) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] error_desc=err_desc) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During handling of the above exception, another exception occurred: 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Traceback (most recent call last): 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] yield resources 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] accel_info=accel_info) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(*args, **kwargs) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] admin_password, network_info, block_device_info) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(self, *args, **kwds) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] should_plug_vifs=False) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'req_state': req_state}) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] time.sleep(sleep_time) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return f(*args, **kwargs) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return self._wait_for_job(job_path) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] error_desc=err_desc) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.264 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.264 4076 3051113294840 MainThread INFO nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Terminating instance 2021-09-10 17:21:41.264 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:21:41.264 4076 3051113294840 MainThread INFO nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Got request to destroy instance 2021-09-10 17:21:41.264 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.vmops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:21:41.289 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:41.289 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "instance-0000007f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:41.289 4076 3051113294840 MainThread INFO nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:21:41.289 4076 3051113294840 MainThread DEBUG nova.compute.claims [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:21:41.289 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:41.399 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:21:41.421 4076 3051113294840 MainThread DEBUG nova.virt.hyperv.hostops [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:21:41.619 4076 3051113294840 MainThread DEBUG nova.compute.provider_tree [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.scheduler.client.report [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:41.652 4076 3051113294840 MainThread ERROR nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Traceback (most recent call last): 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(*args, **kwargs) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] admin_password, network_info, block_device_info) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(self, *args, **kwds) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] should_plug_vifs=False) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'req_state': req_state}) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] time.sleep(sleep_time) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return f(*args, **kwargs) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return self._wait_for_job(job_path) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] error_desc=err_desc) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] During handling of the above exception, another exception occurred: 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Traceback (most recent call last): 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] accel_info=accel_info) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(*args, **kwargs) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] admin_password, network_info, block_device_info) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return function(self, *args, **kwds) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] should_plug_vifs=False) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'req_state': req_state}) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] time.sleep(sleep_time) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self.force_reraise() 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] raise self.value 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return f(*args, **kwargs) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] return self._wait_for_job(job_path) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] error_desc=err_desc) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000007f' failed to start. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Not enough memory in the system to start the virtual machine instance-0000007f with ram size 1024 megabytes. (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 'instance-0000007f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 7E726BB3-5FA2-4188-BD9A-8681613DB624) Error code: 32778. 2021-09-10 17:21:41.652 4076 3051113294840 MainThread TRACE nova.compute.manager [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.compute.utils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] WMI job failed with status 10. Error summary description: 'instance-0000007f' failed to start. Not enough memory in the system to start the virtual machine instance-0000007f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Retry info not present, will not reschedule _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2233 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:21:41.652 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:21:42.145 4076 3051113294840 MainThread DEBUG nova.network.neutron [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:21:42.164 4076 3051113294840 MainThread INFO nova.compute.manager [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] [instance: 4af2d18a-b831-4268-8b2d-64047ec5e874] Took 0.52 seconds to deallocate network for instance. 2021-09-10 17:21:42.399 4076 3051113294840 MainThread INFO nova.scheduler.client.report [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Deleted allocations for instance 4af2d18a-b831-4268-8b2d-64047ec5e874 2021-09-10 17:21:42.399 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [req-ee2ed04d-453a-4c0d-a831-1ded964174b5 5a0c939215da43eba876043c11bbe857 b0e7390862b443e2bdd40ef1bdbc2eb2 - default default] Lock "4af2d18a-b831-4268-8b2d-64047ec5e874" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 21.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:47.802 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:47.802 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:21:47.802 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 17:21:47.898 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:21:47.898 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:21:47.898 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 17:21:48.977 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:21:48.996 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:21:48.996 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:21:48.996 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:53.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:53.973 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:54.816 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:54.816 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:21:55.748 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:56.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:56.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:21:56.804 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:56.804 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:56.804 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:21:56.804 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:21:56.817 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:21:57.008 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2208MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:21:57.008 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:21:57.492 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:21:57.579 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:21:57.630 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:21:57.630 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:21:57.682 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:21:57.763 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:21:57.763 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:21:57.783 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:21:57.996 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:21:58.018 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:21:58.018 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:21:58.018 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 1.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:21:59.003 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:03.785 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:05.773 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:10.041 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:10.056 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:22:10.180 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:10.180 4076 3051113293256 MainThread DEBUG nova.virt.hardware [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:22:10.180 4076 3051113293256 MainThread INFO nova.compute.claims [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Claim successful on node n-h1-808474-1 2021-09-10 17:22:10.344 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:22:10.344 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:22:10.561 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:22:10.578 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:22:10.578 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:10.578 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:22:10.648 4076 3051113293256 MainThread DEBUG nova.compute.utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:22:10.648 4076 3051113164648 GreenThread-116 DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:22:10.648 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:22:10.695 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:22:10.758 4076 3051113293256 MainThread INFO nova.virt.block_device [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Booting with volume 53065bd6-c434-4af7-8c93-3a704bbd9a51 at /dev/sda 2021-09-10 17:22:10.788 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:10.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:22:10.882 4076 3051113164648 GreenThread-116 DEBUG nova.policy [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '82eacb807eb449a28e17290cd4d841d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0aa6ac8af84d7291f244f216bb94b1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:22:10.882 4076 3051113293256 MainThread DEBUG os_brick.utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:22:10.882 4076 3051113293256 MainThread DEBUG os_brick.utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:22:10.882 4076 3051113293256 MainThread DEBUG nova.virt.block_device [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Updating existing volume attachment record: 54e1a025-8a2c-49ee-9a96-6d38724379a5 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:22:11.836 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Successfully created port: 285c0278-c464-409c-8329-43ef6d84ce0c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:22:12.929 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Successfully updated port: 285c0278-c464-409c-8329-43ef6d84ce0c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:22:12.941 4076 3051113164648 GreenThread-116 DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Acquired lock "refresh_cache-df9dee3f-cee2-4374-b88f-571d3da1198c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:22:12.941 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:22:13.039 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Received event network-changed-285c0278-c464-409c-8329-43ef6d84ce0c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:22:13.039 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Refreshing instance network info cache due to event network-changed-285c0278-c464-409c-8329-43ef6d84ce0c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:22:13.066 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:22:13.572 4076 3051113164648 GreenThread-116 DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Updating instance_info_cache with network_info: [{"id": "285c0278-c464-409c-8329-43ef6d84ce0c", "address": "fa:16:3e:fe:31:9f", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap285c0278-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:22:13.602 4076 3051113164648 GreenThread-116 DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Releasing lock "refresh_cache-df9dee3f-cee2-4374-b88f-571d3da1198c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:22:13.602 4076 3051113164648 GreenThread-116 DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Instance network_info: |[{"id": "285c0278-c464-409c-8329-43ef6d84ce0c", "address": "fa:16:3e:fe:31:9f", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap285c0278-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:22:13.602 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-df9dee3f-cee2-4374-b88f-571d3da1198c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:22:13.602 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Refreshing network info cache for port 285c0278-c464-409c-8329-43ef6d84ce0c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:22:14.354 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Updated VIF entry in instance network info cache for port 285c0278-c464-409c-8329-43ef6d84ce0c. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:22:14.354 4076 3051113294136 MainThread DEBUG nova.network.neutron [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Updating instance_info_cache with network_info: [{"id": "285c0278-c464-409c-8329-43ef6d84ce0c", "address": "fa:16:3e:fe:31:9f", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap285c0278-c4", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:22:14.372 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-b48f1fd4-575b-4ff5-aed2-2a97f6b9ed31 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-df9dee3f-cee2-4374-b88f-571d3da1198c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:22:15.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:15.769 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:22:15.774 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:22:16.431 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:22:16.457 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Spawning new instance 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG nova.block_device [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Preparing to wait for external event network-vif-plugged-285c0278-c464-409c-8329-43ef6d84ce0c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:16.467 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Creating VM instance-00000080 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:22:16.901 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.volumeops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51', 'target_lun': 0, 'volume_id': '53065bd6-c434-4af7-8c93-3a704bbd9a51', 'auth_method': 'CHAP', 'auth_username': 'f9Q3UXsa8jpzy5QsH5BW', 'auth_password': '***', 'encrypted': True, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'df9dee3f-cee2-4374-b88f-571d3da1198c', 'attached_at': '', 'detached_at': '', 'volume_id': '53065bd6-c434-4af7-8c93-3a704bbd9a51', 'serial': '53065bd6-c434-4af7-8c93-3a704bbd9a51'} to instance-00000080 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:22:16.901 4076 3051113293256 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51', 'target_lun': 0, 'volume_id': '53065bd6-c434-4af7-8c93-3a704bbd9a51', 'auth_method': 'CHAP', 'auth_username': 'f9Q3UXsa8jpzy5QsH5BW', 'auth_password': '***', 'encrypted': True, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:22:16.901 4076 3051113293256 MainThread INFO os_brick.initiator.windows.iscsi [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:22:16.901 4076 3051113293256 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:22:21.926 4076 3051113293256 MainThread DEBUG os_win.utils.storage.diskutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:22:22.060 4076 3051113293256 MainThread DEBUG os_win.utils.storage.diskutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:22:22.648 4076 3051113293256 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== connect_volume: return (5746ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '36001405B9C943C21DCE4DB791B0E5FB0'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:22:23.063 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:22:28.276 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9e69d9bd-a48c-4d47-8bb2-baa9e0bc34bd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Received event network-vif-plugged-285c0278-c464-409c-8329-43ef6d84ce0c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:22:28.276 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-9e69d9bd-a48c-4d47-8bb2-baa9e0bc34bd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:28.276 4076 3051113164648 MainThread DEBUG oslo_concurrency.lockutils [req-9e69d9bd-a48c-4d47-8bb2-baa9e0bc34bd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:28.276 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9e69d9bd-a48c-4d47-8bb2-baa9e0bc34bd 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Processing event network-vif-plugged-285c0278-c464-409c-8329-43ef6d84ce0c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:22:28.316 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Using config drive for instance 2021-09-10 17:22:28.413 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Creating config drive at C:\openstack\instances\instance-00000080\configdrive.iso 2021-09-10 17:22:28.453 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000080\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp73gyzt42 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:22:28.753 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000080\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp73gyzt42" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:22:29.167 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Setting boot order for instance: instance-00000080: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:22:29.216 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:22:29.237 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '7A1D3B7A-FF09-4547-8DC7-D8400F3662B6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14312), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:29.369 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:29.369 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:29.369 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:29.369 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:29.387 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7A1D3B7A-FF09-4547-8DC7-D8400F3662B6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111390), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:29.387 4076 3051113293256 MainThread DEBUG os_win._utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17164230346679688 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:22:29.387 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9aa520bb-e91a-4628-9eea-ce63e16162c0 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:29.387 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-9aa520bb-e91a-4628-9eea-ce63e16162c0 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:29.432 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9aa520bb-e91a-4628-9eea-ce63e16162c0 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:29.453 4076 3051113164648 MainThread INFO nova.compute.manager [req-9aa520bb-e91a-4628-9eea-ce63e16162c0 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:30.397 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '2F4743E7-D65D-44C8-A547-5F002680F1CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14950), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:30.539 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:30.539 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:30.539 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:30.539 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:30.539 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2F4743E7-D65D-44C8-A547-5F002680F1CD', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110501), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:30.539 4076 3051113293256 MainThread DEBUG os_win._utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.322843074798584 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:22:30.539 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9faa96eb-da69-4aba-a310-a75cb167260d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:30.539 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-9faa96eb-da69-4aba-a310-a75cb167260d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:30.608 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-9faa96eb-da69-4aba-a310-a75cb167260d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:30.634 4076 3051113164648 MainThread INFO nova.compute.manager [req-9faa96eb-da69-4aba-a310-a75cb167260d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:31.554 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'A05D1F9B-9C36-4E1F-B86C-C8162CB82154', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16570), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:31.714 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:31.714 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:31.714 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:31.714 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:31.714 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A05D1F9B-9C36-4E1F-B86C-C8162CB82154', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117668), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:31.714 4076 3051113293256 MainThread DEBUG os_win._utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.4982926845550537 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:22:31.738 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-60e09ff4-de14-4d5e-87c3-ff25a4add00d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:31.738 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-60e09ff4-de14-4d5e-87c3-ff25a4add00d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:31.789 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-60e09ff4-de14-4d5e-87c3-ff25a4add00d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:31.805 4076 3051113164648 MainThread INFO nova.compute.manager [req-60e09ff4-de14-4d5e-87c3-ff25a4add00d - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:32.742 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '73A1BE41-425F-4630-ACC1-9A323C79E936', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15816), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:32.869 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:32.869 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:32.869 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:32.882 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:32.882 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '73A1BE41-425F-4630-ACC1-9A323C79E936', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113372), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:32.882 4076 3051113293256 MainThread DEBUG os_win._utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6667230129241943 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:22:32.899 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-b658b197-36e4-4050-90b0-3585394b1562 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:32.899 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-b658b197-36e4-4050-90b0-3585394b1562 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:32.941 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-b658b197-36e4-4050-90b0-3585394b1562 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:32.970 4076 3051113164648 MainThread INFO nova.compute.manager [req-b658b197-36e4-4050-90b0-3585394b1562 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:33.909 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'A6B6143D-9915-4FAA-AFC8-CAB6A05268A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14684), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:34.039 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:34.039 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:34.039 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:34.039 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:34.062 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-451fbce6-6a78-4d47-9d5e-f8e4081bcc23 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:34.062 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-451fbce6-6a78-4d47-9d5e-f8e4081bcc23 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:34.062 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A6B6143D-9915-4FAA-AFC8-CAB6A05268A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110345), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:34.062 4076 3051113293256 MainThread DEBUG os_win._utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.846099138259888 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:22:34.117 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-451fbce6-6a78-4d47-9d5e-f8e4081bcc23 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:34.117 4076 3051113164648 MainThread INFO nova.compute.manager [req-451fbce6-6a78-4d47-9d5e-f8e4081bcc23 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:35.087 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'A8CACE67-B398-4509-ABE8-6625725C9B68', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15190), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:35.226 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:35.226 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:35.226 4076 3051113164648 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:22:35.226 4076 3051113164648 MainThread INFO nova.compute.manager [-] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] VM Stopped (Lifecycle Event) 2021-09-10 17:22:35.226 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000080\' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000080\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)instance-000000806857257B-57BD-4B57-8A9C-871F0D0860F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A8CACE67-B398-4509-ABE8-6625725C9B68', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109284), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 22, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 22, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\nNot enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)\n\n'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7)", 'ErrorSummaryDescription': "'instance-00000080' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000080.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:22:35.226 4076 3051113293256 MainThread ERROR nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Failed to change vm state of instance-00000080 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. 2021-09-10 17:22:35.226 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Got request to destroy instance 2021-09-10 17:22:35.250 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-064e8031-6d26-4233-b5e4-6d6be4e57ea8 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:22:35.250 4076 3051113164648 MainThread DEBUG nova.virt.hyperv.vmops [req-064e8031-6d26-4233-b5e4-6d6be4e57ea8 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:22:35.304 4076 3051113164648 MainThread DEBUG nova.compute.manager [req-064e8031-6d26-4233-b5e4-6d6be4e57ea8 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:22:35.323 4076 3051113164648 MainThread INFO nova.compute.manager [req-064e8031-6d26-4233-b5e4-6d6be4e57ea8 - - - - -] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:22:37.023 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:22:37.047 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:37.047 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:37.047 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed vm state of instance-00000080 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:22:37.047 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed state of VM instance-00000080 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:22:37.090 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'F52C60A3-720A-44C6-B759-7F727AD48F2F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18485), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:22:37.238 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job succeeded: {'InstanceID': 'F52C60A3-720A-44C6-B759-7F727AD48F2F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29924), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG os_win.utils.pathutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing directory: C:\openstack\instances\instance-00000080 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:37.243 4076 3051113293256 MainThread ERROR nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Traceback (most recent call last): 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(*args, **kwargs) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] admin_password, network_info, block_device_info) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(self, *args, **kwds) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] should_plug_vifs=False) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'req_state': req_state}) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] time.sleep(sleep_time) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return f(*args, **kwargs) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return self._wait_for_job(job_path) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] error_desc=err_desc) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During handling of the above exception, another exception occurred: 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Traceback (most recent call last): 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] yield resources 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] accel_info=accel_info) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(*args, **kwargs) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] admin_password, network_info, block_device_info) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(self, *args, **kwds) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] should_plug_vifs=False) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'req_state': req_state}) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] time.sleep(sleep_time) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return f(*args, **kwargs) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return self._wait_for_job(job_path) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] error_desc=err_desc) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:37.243 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:37.243 4076 3051113293256 MainThread INFO nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Terminating instance 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:22:37.243 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Got request to destroy instance 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51', 'target_lun': 0, 'volume_id': '53065bd6-c434-4af7-8c93-3a704bbd9a51', 'auth_method': 'CHAP', 'auth_username': 'f9Q3UXsa8jpzy5QsH5BW', 'auth_password': '***', 'encrypted': True, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:22:37.243 4076 3051113293256 MainThread DEBUG os_win.utils.storage.diskutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:22:37.360 4076 3051113293256 MainThread DEBUG os_win.utils.storage.diskutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:22:37.372 4076 3051113293256 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:22:37.441 4076 3051113293256 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-53065bd6-c434-4af7-8c93-3a704bbd9a51 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:22:37.441 4076 3051113293256 MainThread DEBUG os_brick.initiator.windows.iscsi [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== disconnect_volume: return (197ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:22:37.441 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:37.441 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000080" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:37.441 4076 3051113293256 MainThread INFO nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Took 0.20 seconds to destroy the instance on the hypervisor. 2021-09-10 17:22:40.288 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-9c2fa97c-1626-4bf3-bf96-47f02f1d266b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Received event network-vif-unplugged-285c0278-c464-409c-8329-43ef6d84ce0c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:22:40.288 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-9c2fa97c-1626-4bf3-bf96-47f02f1d266b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:40.288 4076 3051113294136 MainThread DEBUG oslo_concurrency.lockutils [req-9c2fa97c-1626-4bf3-bf96-47f02f1d266b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:40.316 4076 3051113294136 MainThread DEBUG nova.compute.manager [req-9c2fa97c-1626-4bf3-bf96-47f02f1d266b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] No waiting events found dispatching network-vif-unplugged-285c0278-c464-409c-8329-43ef6d84ce0c pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:22:40.316 4076 3051113294136 MainThread WARNING nova.compute.manager [req-9c2fa97c-1626-4bf3-bf96-47f02f1d266b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Received unexpected event network-vif-unplugged-285c0278-c464-409c-8329-43ef6d84ce0c for instance with vm_state building and task_state spawning. 2021-09-10 17:22:40.835 4076 3051113293256 MainThread INFO nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Took 3.39 seconds to detach 1 volumes for instance. 2021-09-10 17:22:40.835 4076 3051113293256 MainThread DEBUG nova.compute.claims [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:22:40.835 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:40.913 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:22:40.944 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:22:41.152 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:41.169 4076 3051113293256 MainThread ERROR nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Traceback (most recent call last): 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(*args, **kwargs) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] admin_password, network_info, block_device_info) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(self, *args, **kwds) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] should_plug_vifs=False) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'req_state': req_state}) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] time.sleep(sleep_time) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return f(*args, **kwargs) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return self._wait_for_job(job_path) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] error_desc=err_desc) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] During handling of the above exception, another exception occurred: 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Traceback (most recent call last): 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] accel_info=accel_info) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(*args, **kwargs) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] admin_password, network_info, block_device_info) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return function(self, *args, **kwds) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] should_plug_vifs=False) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'req_state': req_state}) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] time.sleep(sleep_time) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self.force_reraise() 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] raise self.value 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return f(*args, **kwargs) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] return self._wait_for_job(job_path) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] error_desc=err_desc) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. 2021-09-10 17:22:41.169 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.compute.utils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Build of instance df9dee3f-cee2-4374-b88f-571d3da1198c was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000080' failed to start. Not enough memory in the system to start the virtual machine instance-00000080. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000080' failed to start. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Not enough memory in the system to start the virtual machine instance-00000080 with ram size 1024 megabytes. (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) 'instance-00000080' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 6857257B-57BD-4B57-8A9C-871F0D0860F7) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:22:41.169 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:22:41.936 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:22:41.970 4076 3051113293256 MainThread INFO nova.compute.manager [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: df9dee3f-cee2-4374-b88f-571d3da1198c] Took 0.81 seconds to deallocate network for instance. 2021-09-10 17:22:42.152 4076 3051113293256 MainThread INFO nova.scheduler.client.report [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Deleted allocations for instance df9dee3f-cee2-4374-b88f-571d3da1198c 2021-09-10 17:22:42.175 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-6dd6b3ca-f9a8-42d0-aafc-88a4afe66762 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "df9dee3f-cee2-4374-b88f-571d3da1198c" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 32.140s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:47.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:47.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:22:47.850 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:22:47.850 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:22:48.335 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:22:48.359 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:22:48.359 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:22:50.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:55.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:55.787 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:55.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:22:56.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:56.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:56.788 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:56.788 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:56.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:22:56.788 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:22:56.816 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:22:57.007 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2198MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:22:57.007 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:22:57.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:22:57.182 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:22:57.182 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:22:57.382 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:22:57.415 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:22:57.415 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:22:57.415 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:22:58.412 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:58.412 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:22:58.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:04.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:21.694 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:21.694 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:23:21.789 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:21.789 4076 3051049643176 MainThread DEBUG nova.virt.hardware [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:23:21.789 4076 3051049643176 MainThread INFO nova.compute.claims [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Claim successful on node n-h1-808474-1 2021-09-10 17:23:21.903 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.hostops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:23:21.936 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.hostops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:23:22.146 4076 3051049643176 MainThread DEBUG nova.compute.provider_tree [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:23:22.161 4076 3051049643176 MainThread DEBUG nova.scheduler.client.report [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:23:22.178 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:22.178 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:23:22.222 4076 3051049643176 MainThread DEBUG nova.compute.utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:23:22.242 4076 3051049643704 GreenThread-117 DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:23:22.242 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:23:22.242 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:23:22.327 4076 3051049643176 MainThread INFO nova.virt.block_device [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Booting with volume 70f42dc2-02c6-4ce3-9c41-d23f15c09a93 at /dev/sda 2021-09-10 17:23:22.366 4076 3051049643704 GreenThread-117 DEBUG nova.policy [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'c27ee1b52b274727b6211034f9223342', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a7f6adcb5eeb4af58d2b79814528b63b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:23:22.412 4076 3051049643176 MainThread DEBUG os_brick.utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:23:22.412 4076 3051049643176 MainThread DEBUG os_brick.utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:23:22.412 4076 3051049643176 MainThread DEBUG nova.virt.block_device [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Updating existing volume attachment record: 5748b531-0889-40f0-9dd3-4066f818b5df _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:23:23.096 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Successfully created port: 96f8159c-f5d7-4667-8ed9-8b6118f0b186 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:23:23.936 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Successfully updated port: 96f8159c-f5d7-4667-8ed9-8b6118f0b186 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:23:23.952 4076 3051049643704 GreenThread-117 DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Acquired lock "refresh_cache-82618a53-c60b-4a16-8224-464db13061d1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:23:23.952 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:23:24.006 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Received event network-changed-96f8159c-f5d7-4667-8ed9-8b6118f0b186 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:23:24.006 4076 3051093169256 MainThread DEBUG nova.compute.manager [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Refreshing instance network info cache due to event network-changed-96f8159c-f5d7-4667-8ed9-8b6118f0b186. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:23:24.052 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:23:24.437 4076 3051049643704 GreenThread-117 DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Updating instance_info_cache with network_info: [{"id": "96f8159c-f5d7-4667-8ed9-8b6118f0b186", "address": "fa:16:3e:72:95:d3", "network": {"id": "394225f6-b2a5-41e0-bc05-3afc686320e7", "bridge": null, "label": "tempest-TestVolumeBackupRestore-402230705-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "a7f6adcb5eeb4af58d2b79814528b63b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap96f8159c-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:23:24.452 4076 3051049643704 GreenThread-117 DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Releasing lock "refresh_cache-82618a53-c60b-4a16-8224-464db13061d1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:23:24.452 4076 3051049643704 GreenThread-117 DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Instance network_info: |[{"id": "96f8159c-f5d7-4667-8ed9-8b6118f0b186", "address": "fa:16:3e:72:95:d3", "network": {"id": "394225f6-b2a5-41e0-bc05-3afc686320e7", "bridge": null, "label": "tempest-TestVolumeBackupRestore-402230705-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "a7f6adcb5eeb4af58d2b79814528b63b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap96f8159c-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:23:24.452 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-82618a53-c60b-4a16-8224-464db13061d1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:23:24.452 4076 3051093169256 MainThread DEBUG nova.network.neutron [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Refreshing network info cache for port 96f8159c-f5d7-4667-8ed9-8b6118f0b186 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:23:25.068 4076 3051093169256 MainThread DEBUG nova.network.neutron [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Updated VIF entry in instance network info cache for port 96f8159c-f5d7-4667-8ed9-8b6118f0b186. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:23:25.068 4076 3051093169256 MainThread DEBUG nova.network.neutron [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Updating instance_info_cache with network_info: [{"id": "96f8159c-f5d7-4667-8ed9-8b6118f0b186", "address": "fa:16:3e:72:95:d3", "network": {"id": "394225f6-b2a5-41e0-bc05-3afc686320e7", "bridge": null, "label": "tempest-TestVolumeBackupRestore-402230705-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "a7f6adcb5eeb4af58d2b79814528b63b", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap96f8159c-f5", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:23:25.078 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [req-9cd8778a-e1a6-4011-bb37-2464dda0702d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-82618a53-c60b-4a16-8224-464db13061d1" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:23:26.596 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:23:26.615 4076 3051049643176 MainThread INFO nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Spawning new instance 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG nova.block_device [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Preparing to wait for external event network-vif-plugged-96f8159c-f5d7-4667-8ed9-8b6118f0b186 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:26.615 4076 3051049643176 MainThread DEBUG os_win.utils.compute.vmutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Creating VM instance-00000081 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:23:27.084 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.volumeops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'target_lun': 0, 'volume_id': '70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'auth_method': 'CHAP', 'auth_username': 'oNjfgyfFJpA46Qn36YH2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '82618a53-c60b-4a16-8224-464db13061d1', 'attached_at': '', 'detached_at': '', 'volume_id': '70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'serial': '70f42dc2-02c6-4ce3-9c41-d23f15c09a93'} to instance-00000081 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:23:27.084 4076 3051049643176 MainThread DEBUG os_brick.initiator.windows.iscsi [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'target_lun': 0, 'volume_id': '70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'auth_method': 'CHAP', 'auth_username': 'oNjfgyfFJpA46Qn36YH2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:23:27.084 4076 3051049643176 MainThread INFO os_brick.initiator.windows.iscsi [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:23:27.084 4076 3051049643176 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:23:32.111 4076 3051049643176 MainThread DEBUG os_win.utils.storage.diskutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:23:32.235 4076 3051049643176 MainThread DEBUG os_win.utils.storage.diskutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:23:32.303 4076 3051049643176 MainThread DEBUG os_brick.initiator.windows.iscsi [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] <== connect_volume: return (5219ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '360014054E1E847474B440889CA9A829B'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:23:32.730 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:23:36.490 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-926d82ed-5b38-4f8a-bf59-78568f080926 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Received event network-vif-plugged-96f8159c-f5d7-4667-8ed9-8b6118f0b186 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:23:36.490 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-926d82ed-5b38-4f8a-bf59-78568f080926 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:36.490 4076 3051049643704 MainThread DEBUG oslo_concurrency.lockutils [req-926d82ed-5b38-4f8a-bf59-78568f080926 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:36.490 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-926d82ed-5b38-4f8a-bf59-78568f080926 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Processing event network-vif-plugged-96f8159c-f5d7-4667-8ed9-8b6118f0b186 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:23:36.519 4076 3051049643176 MainThread INFO nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Using config drive for instance 2021-09-10 17:23:36.706 4076 3051049643176 MainThread INFO nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Creating config drive at C:\openstack\instances\instance-00000081\configdrive.iso 2021-09-10 17:23:36.743 4076 3051049643176 MainThread DEBUG oslo_concurrency.processutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000081\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphrux1bw0 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:23:37.037 4076 3051049643176 MainThread DEBUG oslo_concurrency.processutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000081\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphrux1bw0" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:23:37.444 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Setting boot order for instance: instance-00000081: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:23:37.506 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:23:37.537 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': '3345209B-D853-4F32-BB91-2BA9B01C5651', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19031), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:37.678 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:37.678 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:37.678 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:37.678 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:37.678 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3345209B-D853-4F32-BB91-2BA9B01C5651', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117305), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:37.678 4076 3051049643176 MainThread DEBUG os_win._utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17188358306884766 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:23:37.709 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-4e4513ba-6626-421c-9143-4ff3ca2d8e2c - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:37.709 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-4e4513ba-6626-421c-9143-4ff3ca2d8e2c - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:37.759 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-4e4513ba-6626-421c-9143-4ff3ca2d8e2c - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:37.759 4076 3051049643704 MainThread INFO nova.compute.manager [req-4e4513ba-6626-421c-9143-4ff3ca2d8e2c - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:38.693 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': '451190C1-DABC-44BC-8F8D-627B5DC0D505', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14781), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:38.822 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:38.822 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:38.844 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:38.844 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:38.844 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '451190C1-DABC-44BC-8F8D-627B5DC0D505', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110365), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:38.844 4076 3051049643176 MainThread DEBUG os_win._utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3377346992492676 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:23:38.844 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-fbfafe73-4a36-42df-86e0-38aa978d7c6b - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:38.844 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-fbfafe73-4a36-42df-86e0-38aa978d7c6b - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:38.906 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-fbfafe73-4a36-42df-86e0-38aa978d7c6b - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:38.927 4076 3051049643704 MainThread INFO nova.compute.manager [req-fbfafe73-4a36-42df-86e0-38aa978d7c6b - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:39.881 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': '925D4EAE-E93E-4109-88C5-E74C8A2D984B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14689), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:39.991 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:39.991 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:39.991 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:39.991 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:40.025 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '925D4EAE-E93E-4109-88C5-E74C8A2D984B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110925), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:40.025 4076 3051049643176 MainThread DEBUG os_win._utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5183231830596924 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:23:40.025 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-f9618788-d260-4a43-a908-8475a9c437fe - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:40.025 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-f9618788-d260-4a43-a908-8475a9c437fe - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:40.080 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-f9618788-d260-4a43-a908-8475a9c437fe - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:40.111 4076 3051049643704 MainThread INFO nova.compute.manager [req-f9618788-d260-4a43-a908-8475a9c437fe - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:41.068 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': '69BB2A29-4BAD-4422-85AC-453F7054C546', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14851), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:41.197 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:41.197 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:41.197 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:41.197 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:41.216 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '69BB2A29-4BAD-4422-85AC-453F7054C546', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111329), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:41.216 4076 3051049643176 MainThread DEBUG os_win._utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7098779678344727 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:23:41.229 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-638dbf22-6bfd-4c49-b321-03be9d2ac9e4 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:41.229 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-638dbf22-6bfd-4c49-b321-03be9d2ac9e4 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:41.257 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-638dbf22-6bfd-4c49-b321-03be9d2ac9e4 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:41.302 4076 3051049643704 MainThread INFO nova.compute.manager [req-638dbf22-6bfd-4c49-b321-03be9d2ac9e4 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:42.240 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': 'C5B2EFDF-8076-457F-82DC-BCAEC66917A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=16433), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:42.365 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:42.365 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:42.365 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:42.365 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:42.396 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-7830a4fa-c7ae-4151-a79c-e345c2534dab - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:42.396 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-7830a4fa-c7ae-4151-a79c-e345c2534dab - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:42.396 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C5B2EFDF-8076-457F-82DC-BCAEC66917A4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112381), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:42.396 4076 3051049643176 MainThread DEBUG os_win._utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.889594078063965 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:23:42.431 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-7830a4fa-c7ae-4151-a79c-e345c2534dab - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:42.458 4076 3051049643704 MainThread INFO nova.compute.manager [req-7830a4fa-c7ae-4151-a79c-e345c2534dab - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:43.424 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': 'C8058A87-FFE6-4CFC-8611-8A9EFDC17C7F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15269), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:43.558 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:43.558 4076 3051093169256 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:43.558 4076 3051049643704 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:23:43.558 4076 3051049643704 MainThread INFO nova.compute.manager [-] [instance: 82618a53-c60b-4a16-8224-464db13061d1] VM Stopped (Lifecycle Event) 2021-09-10 17:23:43.572 4076 3051049643176 MainThread ERROR os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job failed: {'RawErrors': (('124\'instance-00000081\' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708112030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA833708110243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000081\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)instance-00000081D7D39B9D-2F7A-4BC3-AE60-134FA8337081%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C8058A87-FFE6-4CFC-8611-8A9EFDC17C7F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115161), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 23, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 23, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\nNot enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)\n\n'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081)", 'ErrorSummaryDescription': "'instance-00000081' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000081.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:23:43.572 4076 3051049643176 MainThread ERROR nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Failed to change vm state of instance-00000081 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. 2021-09-10 17:23:43.572 4076 3051049643176 MainThread INFO nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Got request to destroy instance 2021-09-10 17:23:43.586 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-866b5cee-9d33-494f-9395-1673ee854853 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:23:43.586 4076 3051049643704 MainThread DEBUG nova.virt.hyperv.vmops [req-866b5cee-9d33-494f-9395-1673ee854853 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:23:43.632 4076 3051049643704 MainThread DEBUG nova.compute.manager [req-866b5cee-9d33-494f-9395-1673ee854853 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:23:43.662 4076 3051049643704 MainThread INFO nova.compute.manager [req-866b5cee-9d33-494f-9395-1673ee854853 - - - - -] [instance: 82618a53-c60b-4a16-8224-464db13061d1] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:23:45.382 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:23:45.382 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:45.382 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:45.382 4076 3051049643176 MainThread DEBUG os_win.utils.compute.vmutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Successfully changed vm state of instance-00000081 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:23:45.382 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Successfully changed state of VM instance-00000081 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:23:45.443 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Waiting for WMI job: {'InstanceID': '3CE22BAF-0E30-41C0-A11D-0464E9411AE5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13846), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:23:45.582 4076 3051049643176 MainThread DEBUG os_win.utils.jobutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] WMI job succeeded: {'InstanceID': '3CE22BAF-0E30-41C0-A11D-0464E9411AE5', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30321), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:23:45.582 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:45.582 4076 3051049643176 MainThread DEBUG os_win.utils.pathutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Removing directory: C:\openstack\instances\instance-00000081 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:23:45.582 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:45.582 4076 3051049643176 MainThread ERROR nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Traceback (most recent call last): 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(*args, **kwargs) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] admin_password, network_info, block_device_info) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(self, *args, **kwds) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] should_plug_vifs=False) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'req_state': req_state}) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] time.sleep(sleep_time) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return f(*args, **kwargs) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return self._wait_for_job(job_path) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] error_desc=err_desc) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] During handling of the above exception, another exception occurred: 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Traceback (most recent call last): 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] yield resources 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] accel_info=accel_info) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(*args, **kwargs) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] admin_password, network_info, block_device_info) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(self, *args, **kwds) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] should_plug_vifs=False) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'req_state': req_state}) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] time.sleep(sleep_time) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return f(*args, **kwargs) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return self._wait_for_job(job_path) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] error_desc=err_desc) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:45.582 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:45.593 4076 3051049643176 MainThread INFO nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Terminating instance 2021-09-10 17:23:45.593 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:23:45.593 4076 3051049643176 MainThread INFO nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Got request to destroy instance 2021-09-10 17:23:45.593 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.vmops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:23:45.593 4076 3051049643176 MainThread DEBUG os_brick.initiator.windows.iscsi [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'target_lun': 0, 'volume_id': '70f42dc2-02c6-4ce3-9c41-d23f15c09a93', 'auth_method': 'CHAP', 'auth_username': 'oNjfgyfFJpA46Qn36YH2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:23:45.615 4076 3051049643176 MainThread DEBUG os_win.utils.storage.diskutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:23:45.711 4076 3051049643176 MainThread DEBUG os_win.utils.storage.diskutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:23:45.711 4076 3051049643176 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:23:45.786 4076 3051049643176 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-70f42dc2-02c6-4ce3-9c41-d23f15c09a93 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:23:45.786 4076 3051049643176 MainThread DEBUG os_brick.initiator.windows.iscsi [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] <== disconnect_volume: return (171ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:23:45.786 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:45.786 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "instance-00000081" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:45.786 4076 3051049643176 MainThread INFO nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Took 0.19 seconds to destroy the instance on the hypervisor. 2021-09-10 17:23:48.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:48.767 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:23:48.830 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:23:48.830 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:23:49.068 4076 3051049643176 MainThread INFO nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Took 3.28 seconds to detach 1 volumes for instance. 2021-09-10 17:23:49.068 4076 3051049643176 MainThread DEBUG nova.compute.claims [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:23:49.068 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:49.177 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.hostops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:23:49.189 4076 3051049643176 MainThread DEBUG nova.virt.hyperv.hostops [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:23:49.373 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:23:49.398 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:23:49.398 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:23:49.398 4076 3051049643176 MainThread DEBUG nova.compute.provider_tree [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.scheduler.client.report [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:49.420 4076 3051049643176 MainThread ERROR nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Traceback (most recent call last): 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(*args, **kwargs) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] admin_password, network_info, block_device_info) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(self, *args, **kwds) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] should_plug_vifs=False) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'req_state': req_state}) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] time.sleep(sleep_time) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return f(*args, **kwargs) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return self._wait_for_job(job_path) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] error_desc=err_desc) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] During handling of the above exception, another exception occurred: 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Traceback (most recent call last): 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] accel_info=accel_info) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(*args, **kwargs) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] admin_password, network_info, block_device_info) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return function(self, *args, **kwds) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] should_plug_vifs=False) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'req_state': req_state}) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] time.sleep(sleep_time) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self.force_reraise() 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] raise self.value 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return f(*args, **kwargs) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] return self._wait_for_job(job_path) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] error_desc=err_desc) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. 2021-09-10 17:23:49.420 4076 3051049643176 MainThread TRACE nova.compute.manager [instance: 82618a53-c60b-4a16-8224-464db13061d1] 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.compute.utils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Build of instance 82618a53-c60b-4a16-8224-464db13061d1 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000081' failed to start. Not enough memory in the system to start the virtual machine instance-00000081. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000081' failed to start. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Not enough memory in the system to start the virtual machine instance-00000081 with ram size 1024 megabytes. (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) 'instance-00000081' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D7D39B9D-2F7A-4BC3-AE60-134FA8337081) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:23:49.420 4076 3051049643176 MainThread DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:23:50.105 4076 3051049643176 MainThread DEBUG nova.network.neutron [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:23:50.133 4076 3051049643176 MainThread INFO nova.compute.manager [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] [instance: 82618a53-c60b-4a16-8224-464db13061d1] Took 0.72 seconds to deallocate network for instance. 2021-09-10 17:23:50.340 4076 3051049643176 MainThread INFO nova.scheduler.client.report [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Deleted allocations for instance 82618a53-c60b-4a16-8224-464db13061d1 2021-09-10 17:23:50.380 4076 3051049643176 MainThread DEBUG oslo_concurrency.lockutils [req-76ca5694-adfc-40b3-a88d-ddc50429940a c27ee1b52b274727b6211034f9223342 a7f6adcb5eeb4af58d2b79814528b63b - default default] Lock "82618a53-c60b-4a16-8224-464db13061d1" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 28.687s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:51.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:54.970 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:55.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:56.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:56.789 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:56.814 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:56.814 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:56.814 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:23:56.814 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:23:56.834 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:23:57.037 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2201MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:23:57.037 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:23:57.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:23:57.193 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:23:57.208 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:23:57.413 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:23:57.413 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:23:57.428 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:23:57.428 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:23:58.395 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:58.395 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:23:58.783 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:23:59.739 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:04.786 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:20.202 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:20.213 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:24:20.348 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:20.348 4076 3051093168552 MainThread DEBUG nova.virt.hardware [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:24:20.348 4076 3051093168552 MainThread INFO nova.compute.claims [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Claim successful on node n-h1-808474-1 2021-09-10 17:24:20.454 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:24:20.470 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:24:20.673 4076 3051093168552 MainThread DEBUG nova.compute.provider_tree [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:24:20.673 4076 3051093168552 MainThread DEBUG nova.scheduler.client.report [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:24:20.692 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:20.692 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:24:20.769 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:24:20.769 4076 3051093171368 GreenThread-118 DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:24:20.769 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:24:20.819 4076 3051093168552 MainThread INFO nova.virt.block_device [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Booting with volume 1ea6abf8-e7b9-42f9-b32a-c9e896b83f11 at /dev/sda 2021-09-10 17:24:20.890 4076 3051093171368 GreenThread-118 DEBUG nova.policy [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '82eacb807eb449a28e17290cd4d841d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0aa6ac8af84d7291f244f216bb94b1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:24:20.897 4076 3051093168552 MainThread DEBUG os_brick.utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:24:20.897 4076 3051093168552 MainThread DEBUG os_brick.utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:24:20.897 4076 3051093168552 MainThread DEBUG nova.virt.block_device [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Updating existing volume attachment record: 252e31ae-bd57-4459-8d2a-e0640a76ddf8 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:24:21.495 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Successfully created port: d9de6e39-a838-4e6a-89b0-6f74d1d868bf _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:24:22.319 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Successfully updated port: d9de6e39-a838-4e6a-89b0-6f74d1d868bf _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:24:22.348 4076 3051093171368 GreenThread-118 DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Acquired lock "refresh_cache-2196beef-9303-4ce4-a818-d05db0f89d55" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:24:22.348 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:24:22.379 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Received event network-changed-d9de6e39-a838-4e6a-89b0-6f74d1d868bf external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:24:22.379 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Refreshing instance network info cache due to event network-changed-d9de6e39-a838-4e6a-89b0-6f74d1d868bf. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:24:22.464 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:24:22.845 4076 3051093171368 GreenThread-118 DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Updating instance_info_cache with network_info: [{"id": "d9de6e39-a838-4e6a-89b0-6f74d1d868bf", "address": "fa:16:3e:a4:71:f6", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd9de6e39-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:24:22.864 4076 3051093171368 GreenThread-118 DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Releasing lock "refresh_cache-2196beef-9303-4ce4-a818-d05db0f89d55" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:24:22.864 4076 3051093171368 GreenThread-118 DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Instance network_info: |[{"id": "d9de6e39-a838-4e6a-89b0-6f74d1d868bf", "address": "fa:16:3e:a4:71:f6", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd9de6e39-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:24:22.864 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-2196beef-9303-4ce4-a818-d05db0f89d55" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:24:22.864 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Refreshing network info cache for port d9de6e39-a838-4e6a-89b0-6f74d1d868bf _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:24:23.553 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Updated VIF entry in instance network info cache for port d9de6e39-a838-4e6a-89b0-6f74d1d868bf. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:24:23.553 4076 3051055214664 MainThread DEBUG nova.network.neutron [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Updating instance_info_cache with network_info: [{"id": "d9de6e39-a838-4e6a-89b0-6f74d1d868bf", "address": "fa:16:3e:a4:71:f6", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd9de6e39-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:24:23.569 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-c46595a2-b0d3-497d-bcf8-42ec3846af98 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-2196beef-9303-4ce4-a818-d05db0f89d55" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:24:25.876 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:24:25.891 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Spawning new instance 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG nova.block_device [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Preparing to wait for external event network-vif-plugged-d9de6e39-a838-4e6a-89b0-6f74d1d868bf prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:25.906 4076 3051093168552 MainThread DEBUG os_win.utils.compute.vmutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Creating VM instance-00000082 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:24:26.333 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.volumeops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'target_lun': 0, 'volume_id': '1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'auth_method': 'CHAP', 'auth_username': 'YHQSuHP3q8WfRu6F79dD', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '2196beef-9303-4ce4-a818-d05db0f89d55', 'attached_at': '', 'detached_at': '', 'volume_id': '1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'serial': '1ea6abf8-e7b9-42f9-b32a-c9e896b83f11'} to instance-00000082 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:24:26.333 4076 3051093168552 MainThread DEBUG os_brick.initiator.windows.iscsi [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'target_lun': 0, 'volume_id': '1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'auth_method': 'CHAP', 'auth_username': 'YHQSuHP3q8WfRu6F79dD', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:24:26.333 4076 3051093168552 MainThread INFO os_brick.initiator.windows.iscsi [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:24:26.333 4076 3051093168552 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:24:31.364 4076 3051093168552 MainThread DEBUG os_win.utils.storage.diskutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:24:31.473 4076 3051093168552 MainThread DEBUG os_win.utils.storage.diskutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:24:31.512 4076 3051093168552 MainThread DEBUG os_brick.initiator.windows.iscsi [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== connect_volume: return (5179ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '36001405B2BC857C46FD4C48917F7F8BA'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:24:31.931 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:24:36.801 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-3a8eeece-ec5e-4e5d-a6e9-ada68ca64d1d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Received event network-vif-plugged-d9de6e39-a838-4e6a-89b0-6f74d1d868bf external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:24:36.801 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [req-3a8eeece-ec5e-4e5d-a6e9-ada68ca64d1d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:36.801 4076 3051093171368 MainThread DEBUG oslo_concurrency.lockutils [req-3a8eeece-ec5e-4e5d-a6e9-ada68ca64d1d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:36.801 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-3a8eeece-ec5e-4e5d-a6e9-ada68ca64d1d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Processing event network-vif-plugged-d9de6e39-a838-4e6a-89b0-6f74d1d868bf _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:24:36.837 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Using config drive for instance 2021-09-10 17:24:36.927 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Creating config drive at C:\openstack\instances\instance-00000082\configdrive.iso 2021-09-10 17:24:36.990 4076 3051093168552 MainThread DEBUG oslo_concurrency.processutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000082\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp82wnsuof execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:24:37.312 4076 3051093168552 MainThread DEBUG oslo_concurrency.processutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000082\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp82wnsuof" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:24:37.724 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Setting boot order for instance: instance-00000082: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:24:37.773 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:24:37.804 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '0D14CDE8-900E-4728-B63B-9E4B9DD57DA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18585), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:37.944 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:37.944 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:37.944 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:37.944 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:37.944 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0D14CDE8-900E-4728-B63B-9E4B9DD57DA5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113330), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:37.944 4076 3051093168552 MainThread DEBUG os_win._utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17100071907043457 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:24:37.970 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-5fcb55c5-d7f6-4f15-9edd-1e156a5e46c8 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:37.973 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-5fcb55c5-d7f6-4f15-9edd-1e156a5e46c8 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:38.021 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-5fcb55c5-d7f6-4f15-9edd-1e156a5e46c8 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:38.021 4076 3051093171368 MainThread INFO nova.compute.manager [req-5fcb55c5-d7f6-4f15-9edd-1e156a5e46c8 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:38.973 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '57506378-BEFC-41BA-B900-F3585094F81E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14725), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:39.098 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:39.098 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:39.098 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:39.098 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:39.120 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '57506378-BEFC-41BA-B900-F3585094F81E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117059), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:39.120 4076 3051093168552 MainThread DEBUG os_win._utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3473153114318848 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:24:39.134 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-116d1b6a-53f7-47fe-93dd-3262030010fe - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:39.134 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-116d1b6a-53f7-47fe-93dd-3262030010fe - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:39.176 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-116d1b6a-53f7-47fe-93dd-3262030010fe - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:39.192 4076 3051093171368 MainThread INFO nova.compute.manager [req-116d1b6a-53f7-47fe-93dd-3262030010fe - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:40.156 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '4E3375FF-1158-40F7-B44A-D078099BFBDB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15573), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:40.294 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:40.294 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:40.294 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:40.294 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:40.294 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4E3375FF-1158-40F7-B44A-D078099BFBDB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111274), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:40.294 4076 3051093168552 MainThread DEBUG os_win._utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.521272897720337 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:24:40.312 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-2e28a50a-cb7f-487c-ab1b-ad556c320bc3 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:40.312 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-2e28a50a-cb7f-487c-ab1b-ad556c320bc3 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:40.364 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-2e28a50a-cb7f-487c-ab1b-ad556c320bc3 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:40.385 4076 3051093171368 MainThread INFO nova.compute.manager [req-2e28a50a-cb7f-487c-ab1b-ad556c320bc3 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:41.313 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '62B1B911-F2C4-4A3D-90B9-2367101ABC27', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15123), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:41.449 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:41.449 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:41.461 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:41.461 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:41.461 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '62B1B911-F2C4-4A3D-90B9-2367101ABC27', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108470), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:41.461 4076 3051093168552 MainThread DEBUG os_win._utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.688178300857544 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:24:41.478 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-22c25896-56c6-41cb-a55a-d5f668b37718 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:41.478 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-22c25896-56c6-41cb-a55a-d5f668b37718 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:41.513 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-22c25896-56c6-41cb-a55a-d5f668b37718 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:41.529 4076 3051093171368 MainThread INFO nova.compute.manager [req-22c25896-56c6-41cb-a55a-d5f668b37718 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:42.488 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'A0FD767F-AD9E-4D2F-8EA5-B2B893851F3E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15008), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:42.615 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:42.615 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:42.615 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:42.615 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:42.615 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A0FD767F-AD9E-4D2F-8EA5-B2B893851F3E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112068), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:42.615 4076 3051093168552 MainThread DEBUG os_win._utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.842081069946289 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:24:42.645 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-90fe8f6c-a900-4656-bda8-07f3ae178c88 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:42.645 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-90fe8f6c-a900-4656-bda8-07f3ae178c88 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:42.692 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-90fe8f6c-a900-4656-bda8-07f3ae178c88 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:42.713 4076 3051093171368 MainThread INFO nova.compute.manager [req-90fe8f6c-a900-4656-bda8-07f3ae178c88 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:43.647 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '639DFBF5-B336-4108-8B0F-7B476D0299D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15226), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:43.773 4076 3051093171368 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:24:43.773 4076 3051093171368 MainThread INFO nova.compute.manager [-] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] VM Stopped (Lifecycle Event) 2021-09-10 17:24:43.773 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:43.773 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:43.773 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-d9936810-b75d-40ff-ba23-5cbc6b59c970 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:24:43.773 4076 3051093171368 MainThread DEBUG nova.virt.hyperv.vmops [req-d9936810-b75d-40ff-ba23-5cbc6b59c970 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:24:43.801 4076 3051093168552 MainThread ERROR os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000082\' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000082\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)instance-000000821725F27F-6A6E-4149-AF59-A7A19E6FA89B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '639DFBF5-B336-4108-8B0F-7B476D0299D3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108711), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 24, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 24, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\nNot enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)\n\n'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B)", 'ErrorSummaryDescription': "'instance-00000082' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000082.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:24:43.801 4076 3051093168552 MainThread ERROR nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Failed to change vm state of instance-00000082 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. 2021-09-10 17:24:43.801 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Got request to destroy instance 2021-09-10 17:24:43.835 4076 3051093171368 MainThread DEBUG nova.compute.manager [req-d9936810-b75d-40ff-ba23-5cbc6b59c970 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:24:43.853 4076 3051093171368 MainThread INFO nova.compute.manager [req-d9936810-b75d-40ff-ba23-5cbc6b59c970 - - - - -] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:24:45.588 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:24:45.588 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:45.588 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:45.612 4076 3051093168552 MainThread DEBUG os_win.utils.compute.vmutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed vm state of instance-00000082 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:24:45.612 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed state of VM instance-00000082 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:24:45.664 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'B057A2F8-5DE8-450D-B403-0BA4B646510B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13842), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:24:45.799 4076 3051093168552 MainThread DEBUG os_win.utils.jobutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job succeeded: {'InstanceID': 'B057A2F8-5DE8-450D-B403-0BA4B646510B', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29582), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:24:45.799 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:45.799 4076 3051093168552 MainThread DEBUG os_win.utils.pathutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing directory: C:\openstack\instances\instance-00000082 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:24:45.799 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:45.799 4076 3051093168552 MainThread ERROR nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Traceback (most recent call last): 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(*args, **kwargs) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] admin_password, network_info, block_device_info) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(self, *args, **kwds) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] should_plug_vifs=False) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'req_state': req_state}) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] time.sleep(sleep_time) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return f(*args, **kwargs) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return self._wait_for_job(job_path) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] error_desc=err_desc) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During handling of the above exception, another exception occurred: 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Traceback (most recent call last): 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] yield resources 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] accel_info=accel_info) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(*args, **kwargs) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] admin_password, network_info, block_device_info) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(self, *args, **kwds) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] should_plug_vifs=False) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'req_state': req_state}) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] time.sleep(sleep_time) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return f(*args, **kwargs) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return self._wait_for_job(job_path) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] error_desc=err_desc) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:45.799 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:45.799 4076 3051093168552 MainThread INFO nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Terminating instance 2021-09-10 17:24:45.808 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:24:45.808 4076 3051093168552 MainThread INFO nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Got request to destroy instance 2021-09-10 17:24:45.818 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:24:45.818 4076 3051093168552 MainThread DEBUG os_brick.initiator.windows.iscsi [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'target_lun': 0, 'volume_id': '1ea6abf8-e7b9-42f9-b32a-c9e896b83f11', 'auth_method': 'CHAP', 'auth_username': 'YHQSuHP3q8WfRu6F79dD', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:24:45.818 4076 3051093168552 MainThread DEBUG os_win.utils.storage.diskutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:24:45.919 4076 3051093168552 MainThread DEBUG os_win.utils.storage.diskutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:24:45.929 4076 3051093168552 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:24:45.993 4076 3051093168552 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-1ea6abf8-e7b9-42f9-b32a-c9e896b83f11 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:24:45.993 4076 3051093168552 MainThread DEBUG os_brick.initiator.windows.iscsi [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== disconnect_volume: return (174ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:24:45.993 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:45.993 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000082" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:45.993 4076 3051093168552 MainThread INFO nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Took 0.19 seconds to destroy the instance on the hypervisor. 2021-09-10 17:24:49.227 4076 3051093168552 MainThread INFO nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Took 3.23 seconds to detach 1 volumes for instance. 2021-09-10 17:24:49.227 4076 3051093168552 MainThread DEBUG nova.compute.claims [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:24:49.227 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:49.316 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:24:49.332 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.hostops [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:24:49.538 4076 3051093168552 MainThread DEBUG nova.compute.provider_tree [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG nova.scheduler.client.report [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:49.552 4076 3051093168552 MainThread ERROR nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Traceback (most recent call last): 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(*args, **kwargs) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] admin_password, network_info, block_device_info) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(self, *args, **kwds) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] should_plug_vifs=False) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'req_state': req_state}) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] time.sleep(sleep_time) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return f(*args, **kwargs) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return self._wait_for_job(job_path) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] error_desc=err_desc) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] During handling of the above exception, another exception occurred: 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Traceback (most recent call last): 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] accel_info=accel_info) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(*args, **kwargs) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] admin_password, network_info, block_device_info) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return function(self, *args, **kwds) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] should_plug_vifs=False) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'req_state': req_state}) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] time.sleep(sleep_time) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self.force_reraise() 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] raise self.value 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return f(*args, **kwargs) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] return self._wait_for_job(job_path) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] error_desc=err_desc) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. 2021-09-10 17:24:49.552 4076 3051093168552 MainThread TRACE nova.compute.manager [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG nova.compute.utils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Build of instance 2196beef-9303-4ce4-a818-d05db0f89d55 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000082' failed to start. Not enough memory in the system to start the virtual machine instance-00000082. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000082' failed to start. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Not enough memory in the system to start the virtual machine instance-00000082 with ram size 1024 megabytes. (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) 'instance-00000082' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1725F27F-6A6E-4149-AF59-A7A19E6FA89B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:24:49.552 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:24:49.577 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:24:49.577 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:24:50.103 4076 3051093168552 MainThread DEBUG nova.network.neutron [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:24:50.124 4076 3051093168552 MainThread INFO nova.compute.manager [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 2196beef-9303-4ce4-a818-d05db0f89d55] Took 0.55 seconds to deallocate network for instance. 2021-09-10 17:24:50.257 4076 3051093168552 MainThread INFO nova.scheduler.client.report [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Deleted allocations for instance 2196beef-9303-4ce4-a818-d05db0f89d55 2021-09-10 17:24:50.316 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-5846f5b0-d13e-415b-b64d-b6ced1b2ce69 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "2196beef-9303-4ce4-a818-d05db0f89d55" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.109s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:50.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:50.776 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:24:50.977 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:24:50.977 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:24:51.529 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:24:51.537 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:24:51.537 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:24:51.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:56.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:57.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:57.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:57.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:24:58.775 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:24:58.791 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:58.791 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:24:58.791 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:24:58.791 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:24:58.791 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:24:58.988 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2206MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:24:58.988 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:24:59.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:24:59.179 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:24:59.196 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:24:59.400 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:24:59.413 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:24:59.413 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:24:59.413 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:00.395 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:00.395 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:00.426 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:06.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:11.035 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:11.057 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:25:11.141 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:11.144 4076 3051055217128 MainThread DEBUG nova.virt.hardware [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:25:11.144 4076 3051055217128 MainThread INFO nova.compute.claims [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Claim successful on node n-h1-808474-1 2021-09-10 17:25:11.246 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:25:11.260 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:25:11.462 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:25:11.462 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:25:11.462 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:11.462 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:25:11.537 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:25:11.537 4076 3051093168552 GreenThread-119 DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:25:11.537 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:25:11.537 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:25:11.613 4076 3051055217128 MainThread INFO nova.virt.block_device [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Booting with volume 5f36efd2-4359-43b8-bbad-c199178f098b at /dev/sda 2021-09-10 17:25:11.661 4076 3051093168552 GreenThread-119 DEBUG nova.policy [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '82eacb807eb449a28e17290cd4d841d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0aa6ac8af84d7291f244f216bb94b1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:25:11.675 4076 3051055217128 MainThread DEBUG os_brick.utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:25:11.675 4076 3051055217128 MainThread DEBUG os_brick.utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:25:11.675 4076 3051055217128 MainThread DEBUG nova.virt.block_device [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Updating existing volume attachment record: 986de9bf-669b-420f-9c73-fd69071f68e2 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:25:12.302 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Successfully created port: 0dd5c671-dc4c-4d64-92fd-71b99964cf09 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:25:12.927 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Successfully updated port: 0dd5c671-dc4c-4d64-92fd-71b99964cf09 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:25:12.942 4076 3051093168552 GreenThread-119 DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Acquired lock "refresh_cache-996ac19a-6395-455e-85e7-1fc5add701a3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:25:12.942 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:25:12.988 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Received event network-changed-0dd5c671-dc4c-4d64-92fd-71b99964cf09 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:25:12.988 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Refreshing instance network info cache due to event network-changed-0dd5c671-dc4c-4d64-92fd-71b99964cf09. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:25:13.067 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:25:13.488 4076 3051093168552 GreenThread-119 DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Updating instance_info_cache with network_info: [{"id": "0dd5c671-dc4c-4d64-92fd-71b99964cf09", "address": "fa:16:3e:05:06:04", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0dd5c671-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:25:13.489 4076 3051093168552 GreenThread-119 DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Releasing lock "refresh_cache-996ac19a-6395-455e-85e7-1fc5add701a3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:25:13.489 4076 3051093168552 GreenThread-119 DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Instance network_info: |[{"id": "0dd5c671-dc4c-4d64-92fd-71b99964cf09", "address": "fa:16:3e:05:06:04", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0dd5c671-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:25:13.489 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-996ac19a-6395-455e-85e7-1fc5add701a3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:25:13.489 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Refreshing network info cache for port 0dd5c671-dc4c-4d64-92fd-71b99964cf09 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:25:14.015 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Updated VIF entry in instance network info cache for port 0dd5c671-dc4c-4d64-92fd-71b99964cf09. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:25:14.015 4076 3051113161480 MainThread DEBUG nova.network.neutron [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Updating instance_info_cache with network_info: [{"id": "0dd5c671-dc4c-4d64-92fd-71b99964cf09", "address": "fa:16:3e:05:06:04", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0dd5c671-dc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:25:14.015 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-8c3ed5e3-951c-4cb3-8721-b2cb481d0afb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-996ac19a-6395-455e-85e7-1fc5add701a3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:25:15.784 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:25:15.800 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Spawning new instance 2021-09-10 17:25:15.800 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:15.800 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:15.800 4076 3051055217128 MainThread DEBUG nova.block_device [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:25:15.800 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Preparing to wait for external event network-vif-plugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:25:15.816 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:15.816 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:15.816 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Creating VM instance-00000083 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:25:16.238 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.volumeops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b', 'target_lun': 0, 'volume_id': '5f36efd2-4359-43b8-bbad-c199178f098b', 'auth_method': 'CHAP', 'auth_username': 'DhfLE9qb4MypNwLEAe2n', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '996ac19a-6395-455e-85e7-1fc5add701a3', 'attached_at': '', 'detached_at': '', 'volume_id': '5f36efd2-4359-43b8-bbad-c199178f098b', 'serial': '5f36efd2-4359-43b8-bbad-c199178f098b'} to instance-00000083 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:25:16.238 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b', 'target_lun': 0, 'volume_id': '5f36efd2-4359-43b8-bbad-c199178f098b', 'auth_method': 'CHAP', 'auth_username': 'DhfLE9qb4MypNwLEAe2n', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:25:16.238 4076 3051055217128 MainThread INFO os_brick.initiator.windows.iscsi [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:25:16.238 4076 3051055217128 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:25:21.253 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:25:21.365 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:25:21.409 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== connect_volume: return (5171ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '360014059E0CD8D083F944ED9CB64853B'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:25:21.829 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:25:25.066 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-e723f843-4580-4f03-9195-3dda205fb327 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Received event network-vif-plugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:25:25.066 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-e723f843-4580-4f03-9195-3dda205fb327 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:25.066 4076 3051093168552 MainThread DEBUG oslo_concurrency.lockutils [req-e723f843-4580-4f03-9195-3dda205fb327 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:25.066 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-e723f843-4580-4f03-9195-3dda205fb327 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Processing event network-vif-plugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:25:25.101 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Using config drive for instance 2021-09-10 17:25:25.206 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Creating config drive at C:\openstack\instances\instance-00000083\configdrive.iso 2021-09-10 17:25:25.240 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000083\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_5ewx8u7 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:25:25.612 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000083\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_5ewx8u7" returned: 0 in 0.375s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:25:25.991 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Setting boot order for instance: instance-00000083: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:25:26.050 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:25:26.081 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '80AF3D10-F22A-4126-B587-36E2B482929B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14792), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:26.191 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:26.191 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:26.191 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:26.191 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:26.222 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '80AF3D10-F22A-4126-B587-36E2B482929B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112478), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:26.222 4076 3051055217128 MainThread DEBUG os_win._utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1721200942993164 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:25:26.222 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c8872585-d3c6-4ac5-bab7-82d3444b46e0 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:26.239 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-c8872585-d3c6-4ac5-bab7-82d3444b46e0 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:26.284 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-c8872585-d3c6-4ac5-bab7-82d3444b46e0 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:26.303 4076 3051093168552 MainThread INFO nova.compute.manager [req-c8872585-d3c6-4ac5-bab7-82d3444b46e0 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:27.253 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '93F25356-EA68-4A96-922D-B8DC290C0F50', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15366), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:27.381 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:27.381 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:27.381 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:27.381 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:27.396 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '93F25356-EA68-4A96-922D-B8DC290C0F50', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109314), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:27.396 4076 3051055217128 MainThread DEBUG os_win._utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.345993995666504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:25:27.396 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-f1e06e17-1ced-466b-9f05-333b482a613e - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:27.396 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-f1e06e17-1ced-466b-9f05-333b482a613e - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:27.443 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-f1e06e17-1ced-466b-9f05-333b482a613e - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:27.474 4076 3051093168552 MainThread INFO nova.compute.manager [req-f1e06e17-1ced-466b-9f05-333b482a613e - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:28.424 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'C1E56FA0-3A49-4D4C-A12F-5B87BDAF8464', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15045), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:28.554 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:28.554 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:28.554 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:28.554 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:28.572 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-66d4ff3d-4d6a-42b8-bd3f-a60946f2661a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:28.572 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-66d4ff3d-4d6a-42b8-bd3f-a60946f2661a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:28.572 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C1E56FA0-3A49-4D4C-A12F-5B87BDAF8464', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108793), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:28.572 4076 3051055217128 MainThread DEBUG os_win._utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.522510290145874 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:25:28.615 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-66d4ff3d-4d6a-42b8-bd3f-a60946f2661a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:28.631 4076 3051093168552 MainThread INFO nova.compute.manager [req-66d4ff3d-4d6a-42b8-bd3f-a60946f2661a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:29.599 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'C377CACD-0600-4278-931B-9E5F06070D8E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15637), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:29.745 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:29.745 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:29.745 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:29.745 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:29.745 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C377CACD-0600-4278-931B-9E5F06070D8E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112784), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:29.745 4076 3051055217128 MainThread DEBUG os_win._utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6946732997894287 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:25:29.769 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-fcefffdd-37bd-4505-b6c5-f6ad8df64cc1 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:29.769 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-fcefffdd-37bd-4505-b6c5-f6ad8df64cc1 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:29.816 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-fcefffdd-37bd-4505-b6c5-f6ad8df64cc1 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:29.816 4076 3051093168552 MainThread INFO nova.compute.manager [req-fcefffdd-37bd-4505-b6c5-f6ad8df64cc1 - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:30.759 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '0B6E1516-8ED8-4552-B23C-F899C354D7CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14933), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:30.895 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:30.895 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:30.895 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:30.895 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:30.895 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0B6E1516-8ED8-4552-B23C-F899C354D7CB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110268), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 30), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 30), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:30.895 4076 3051055217128 MainThread DEBUG os_win._utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.845432996749878 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:25:30.895 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-9456b6b8-423c-4a7e-a1e2-8810a796649a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:30.895 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-9456b6b8-423c-4a7e-a1e2-8810a796649a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:30.961 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-9456b6b8-423c-4a7e-a1e2-8810a796649a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:30.991 4076 3051093168552 MainThread INFO nova.compute.manager [req-9456b6b8-423c-4a7e-a1e2-8810a796649a - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:31.925 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '17AC10F8-9B50-490D-A44C-79631EDA7983', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14515), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:32.055 4076 3051093168552 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:25:32.055 4076 3051093168552 MainThread INFO nova.compute.manager [-] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] VM Stopped (Lifecycle Event) 2021-09-10 17:25:32.055 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:32.055 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:32.055 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000083\' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000083\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)instance-000000839A60EA34-FEDD-4B44-9E5B-01B225822ACC%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '17AC10F8-9B50-490D-A44C-79631EDA7983', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110695), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 25, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 25, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\nNot enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)\n\n'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC)", 'ErrorSummaryDescription': "'instance-00000083' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000083.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:25:32.078 4076 3051055217128 MainThread ERROR nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Failed to change vm state of instance-00000083 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. 2021-09-10 17:25:32.078 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Got request to destroy instance 2021-09-10 17:25:32.082 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5a33ab09-0fef-4a86-bc6f-af6c7a7ef0be - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:25:32.082 4076 3051093168552 MainThread DEBUG nova.virt.hyperv.vmops [req-5a33ab09-0fef-4a86-bc6f-af6c7a7ef0be - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:25:32.130 4076 3051093168552 MainThread DEBUG nova.compute.manager [req-5a33ab09-0fef-4a86-bc6f-af6c7a7ef0be - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:25:32.154 4076 3051093168552 MainThread INFO nova.compute.manager [req-5a33ab09-0fef-4a86-bc6f-af6c7a7ef0be - - - - -] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:25:33.849 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:25:33.849 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:33.849 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:33.883 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed vm state of instance-00000083 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:25:33.883 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed state of VM instance-00000083 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:25:33.928 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '16E8A54E-71C0-4749-9535-BC379729815C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13933), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:25:34.068 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job succeeded: {'InstanceID': '16E8A54E-71C0-4749-9535-BC379729815C', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30237), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:25:34.068 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:34.068 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing directory: C:\openstack\instances\instance-00000083 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:25:34.068 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:34.068 4076 3051055217128 MainThread ERROR nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Traceback (most recent call last): 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(*args, **kwargs) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] admin_password, network_info, block_device_info) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(self, *args, **kwds) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] should_plug_vifs=False) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'req_state': req_state}) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] time.sleep(sleep_time) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return f(*args, **kwargs) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return self._wait_for_job(job_path) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] error_desc=err_desc) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During handling of the above exception, another exception occurred: 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Traceback (most recent call last): 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] yield resources 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] accel_info=accel_info) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(*args, **kwargs) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] admin_password, network_info, block_device_info) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(self, *args, **kwds) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] should_plug_vifs=False) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'req_state': req_state}) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] time.sleep(sleep_time) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return f(*args, **kwargs) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return self._wait_for_job(job_path) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] error_desc=err_desc) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:34.068 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:34.089 4076 3051055217128 MainThread INFO nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Terminating instance 2021-09-10 17:25:34.089 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:25:34.089 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Got request to destroy instance 2021-09-10 17:25:34.103 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:25:34.103 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b', 'target_lun': 0, 'volume_id': '5f36efd2-4359-43b8-bbad-c199178f098b', 'auth_method': 'CHAP', 'auth_username': 'DhfLE9qb4MypNwLEAe2n', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:25:34.103 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:25:34.193 4076 3051055217128 MainThread DEBUG os_win.utils.storage.diskutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:25:34.209 4076 3051055217128 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:25:34.278 4076 3051055217128 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-5f36efd2-4359-43b8-bbad-c199178f098b _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:25:34.287 4076 3051055217128 MainThread DEBUG os_brick.initiator.windows.iscsi [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== disconnect_volume: return (183ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:25:34.287 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:34.287 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000083" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:34.287 4076 3051055217128 MainThread INFO nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Took 0.20 seconds to destroy the instance on the hypervisor. 2021-09-10 17:25:37.083 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-4816c2a3-7724-4f7f-9717-fb0432b9d1ff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Received event network-vif-unplugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:25:37.083 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-4816c2a3-7724-4f7f-9717-fb0432b9d1ff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:37.083 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [req-4816c2a3-7724-4f7f-9717-fb0432b9d1ff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:37.083 4076 3051113161480 MainThread DEBUG nova.compute.manager [req-4816c2a3-7724-4f7f-9717-fb0432b9d1ff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] No waiting events found dispatching network-vif-unplugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:25:37.083 4076 3051113161480 MainThread WARNING nova.compute.manager [req-4816c2a3-7724-4f7f-9717-fb0432b9d1ff 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Received unexpected event network-vif-unplugged-0dd5c671-dc4c-4d64-92fd-71b99964cf09 for instance with vm_state building and task_state spawning. 2021-09-10 17:25:37.506 4076 3051055217128 MainThread INFO nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Took 3.22 seconds to detach 1 volumes for instance. 2021-09-10 17:25:37.506 4076 3051055217128 MainThread DEBUG nova.compute.claims [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:25:37.506 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:25:37.612 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:25:37.628 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:25:37.838 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:25:37.838 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:25:37.838 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.329s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:37.838 4076 3051055217128 MainThread ERROR nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Traceback (most recent call last): 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(*args, **kwargs) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] admin_password, network_info, block_device_info) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(self, *args, **kwds) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] should_plug_vifs=False) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'req_state': req_state}) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] time.sleep(sleep_time) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return f(*args, **kwargs) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return self._wait_for_job(job_path) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] error_desc=err_desc) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] During handling of the above exception, another exception occurred: 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Traceback (most recent call last): 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] accel_info=accel_info) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(*args, **kwargs) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] admin_password, network_info, block_device_info) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return function(self, *args, **kwds) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] should_plug_vifs=False) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'req_state': req_state}) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] time.sleep(sleep_time) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self.force_reraise() 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] raise self.value 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return f(*args, **kwargs) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] return self._wait_for_job(job_path) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] error_desc=err_desc) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. 2021-09-10 17:25:37.838 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] 2021-09-10 17:25:37.858 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:25:37.858 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Build of instance 996ac19a-6395-455e-85e7-1fc5add701a3 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000083' failed to start. Not enough memory in the system to start the virtual machine instance-00000083. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000083' failed to start. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Not enough memory in the system to start the virtual machine instance-00000083 with ram size 1024 megabytes. (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) 'instance-00000083' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9A60EA34-FEDD-4B44-9E5B-01B225822ACC) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:25:37.863 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:25:37.863 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:25:37.863 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:25:37.863 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:25:38.373 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:25:38.380 4076 3051055217128 MainThread INFO nova.compute.manager [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 996ac19a-6395-455e-85e7-1fc5add701a3] Took 0.51 seconds to deallocate network for instance. 2021-09-10 17:25:38.552 4076 3051055217128 MainThread INFO nova.scheduler.client.report [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Deleted allocations for instance 996ac19a-6395-455e-85e7-1fc5add701a3 2021-09-10 17:25:38.584 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-8747b1e5-700e-4b27-978d-57ba2007b70d 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "996ac19a-6395-455e-85e7-1fc5add701a3" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.547s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:25:51.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:51.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:25:51.846 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:25:51.846 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:25:52.346 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:25:52.363 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:25:52.363 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:25:52.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:56.772 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:57.784 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:58.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:58.774 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:25:59.740 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:59.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:25:59.980 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:00.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:00.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:00.800 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:00.800 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:26:00.800 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:26:00.826 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:26:01.052 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2198MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:26:01.052 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.168 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.190 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.190 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.190 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.190 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:26:01.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:26:01.193 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:26:01.193 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:26:01.235 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:26:01.445 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:26:01.457 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:26:01.457 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:26:01.457 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:09.435 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:21.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_image_cache_manager_pass run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:21.758 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use..do_register_storage_use" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:21.758 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "storage-registry-lock" released by "nova.virt.storage_users.register_storage_use..do_register_storage_use" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:21.758 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users..do_get_storage_users" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:21.758 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "storage-registry-lock" released by "nova.virt.storage_users.get_storage_users..do_get_storage_users" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:27.207 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:27.223 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:26:27.319 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:27.319 4076 3051113163944 MainThread DEBUG nova.virt.hardware [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:26:27.319 4076 3051113163944 MainThread INFO nova.compute.claims [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Claim successful on node n-h1-808474-1 2021-09-10 17:26:27.411 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:26:27.444 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:26:27.633 4076 3051113163944 MainThread DEBUG nova.compute.provider_tree [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:26:27.650 4076 3051113163944 MainThread DEBUG nova.scheduler.client.report [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:26:27.650 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:27.650 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:26:27.723 4076 3051113163944 MainThread DEBUG nova.compute.utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:26:27.723 4076 3051055217128 GreenThread-120 DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:26:27.723 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:26:27.723 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:26:27.785 4076 3051113163944 MainThread INFO nova.virt.block_device [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Booting with volume 1379018d-523d-44fa-961c-1c45d19b8980 at /dev/sda 2021-09-10 17:26:27.830 4076 3051055217128 GreenThread-120 DEBUG nova.policy [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '82eacb807eb449a28e17290cd4d841d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8d0aa6ac8af84d7291f244f216bb94b1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:26:27.861 4076 3051113163944 MainThread DEBUG os_brick.utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:26:27.861 4076 3051113163944 MainThread DEBUG os_brick.utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:26:27.861 4076 3051113163944 MainThread DEBUG nova.virt.block_device [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Updating existing volume attachment record: c69f7617-f451-4061-aecf-c426536f9169 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:26:28.470 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Successfully created port: 674c16b2-c09e-42c8-9819-4931bce2b4da _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:26:29.162 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Successfully updated port: 674c16b2-c09e-42c8-9819-4931bce2b4da _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:26:29.187 4076 3051055217128 GreenThread-120 DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Acquired lock "refresh_cache-58aa12fc-aae6-43d2-8479-9e7a28651c3b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:26:29.187 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:26:29.272 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:26:29.315 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Received event network-changed-674c16b2-c09e-42c8-9819-4931bce2b4da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:26:29.315 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Refreshing instance network info cache due to event network-changed-674c16b2-c09e-42c8-9819-4931bce2b4da. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:26:29.832 4076 3051055217128 GreenThread-120 DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Updating instance_info_cache with network_info: [{"id": "674c16b2-c09e-42c8-9819-4931bce2b4da", "address": "fa:16:3e:e0:5a:9d", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap674c16b2-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:26:29.864 4076 3051055217128 GreenThread-120 DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Releasing lock "refresh_cache-58aa12fc-aae6-43d2-8479-9e7a28651c3b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:26:29.864 4076 3051055217128 GreenThread-120 DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Instance network_info: |[{"id": "674c16b2-c09e-42c8-9819-4931bce2b4da", "address": "fa:16:3e:e0:5a:9d", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap674c16b2-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:26:29.864 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-58aa12fc-aae6-43d2-8479-9e7a28651c3b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:26:29.864 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Refreshing network info cache for port 674c16b2-c09e-42c8-9819-4931bce2b4da _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:26:30.394 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Updated VIF entry in instance network info cache for port 674c16b2-c09e-42c8-9819-4931bce2b4da. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:26:30.394 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Updating instance_info_cache with network_info: [{"id": "674c16b2-c09e-42c8-9819-4931bce2b4da", "address": "fa:16:3e:e0:5a:9d", "network": {"id": "385c7b14-693f-45e3-86cf-8a54c842098b", "bridge": null, "label": "tempest-TestVolumeBootPattern-577893151-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "8d0aa6ac8af84d7291f244f216bb94b1", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap674c16b2-c0", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:26:30.408 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-4b887e93-9fa8-449c-a67c-cca7413676bb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-58aa12fc-aae6-43d2-8479-9e7a28651c3b" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:26:31.911 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:26:31.911 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Spawning new instance 2021-09-10 17:26:31.911 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG nova.block_device [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Preparing to wait for external event network-vif-plugged-674c16b2-c09e-42c8-9819-4931bce2b4da prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:31.938 4076 3051113163944 MainThread DEBUG os_win.utils.compute.vmutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Creating VM instance-00000084 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:26:32.377 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.volumeops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980', 'target_lun': 0, 'volume_id': '1379018d-523d-44fa-961c-1c45d19b8980', 'auth_method': 'CHAP', 'auth_username': 'ZhSmYdnG6t2N8fdpK8zj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': '58aa12fc-aae6-43d2-8479-9e7a28651c3b', 'attached_at': '', 'detached_at': '', 'volume_id': '1379018d-523d-44fa-961c-1c45d19b8980', 'serial': '1379018d-523d-44fa-961c-1c45d19b8980'} to instance-00000084 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:26:32.377 4076 3051113163944 MainThread DEBUG os_brick.initiator.windows.iscsi [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980', 'target_lun': 0, 'volume_id': '1379018d-523d-44fa-961c-1c45d19b8980', 'auth_method': 'CHAP', 'auth_username': 'ZhSmYdnG6t2N8fdpK8zj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:26:32.377 4076 3051113163944 MainThread INFO os_brick.initiator.windows.iscsi [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980 on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:26:32.377 4076 3051113163944 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980 login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:26:37.386 4076 3051113163944 MainThread DEBUG os_win.utils.storage.diskutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:26:37.513 4076 3051113163944 MainThread DEBUG os_win.utils.storage.diskutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:26:37.551 4076 3051113163944 MainThread DEBUG os_brick.initiator.windows.iscsi [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== connect_volume: return (5174ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '360014054590BE021134444F8E8BE2221'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:26:37.979 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:26:41.424 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-08e68ad5-7aac-4c1d-a9bf-01eeaf11f2ae 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Received event network-vif-plugged-674c16b2-c09e-42c8-9819-4931bce2b4da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:26:41.424 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-08e68ad5-7aac-4c1d-a9bf-01eeaf11f2ae 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:41.424 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-08e68ad5-7aac-4c1d-a9bf-01eeaf11f2ae 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:41.424 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-08e68ad5-7aac-4c1d-a9bf-01eeaf11f2ae 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Processing event network-vif-plugged-674c16b2-c09e-42c8-9819-4931bce2b4da _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:26:41.457 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Using config drive for instance 2021-09-10 17:26:41.564 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Creating config drive at C:\openstack\instances\instance-00000084\configdrive.iso 2021-09-10 17:26:41.603 4076 3051113163944 MainThread DEBUG oslo_concurrency.processutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000084\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprlxy9tue execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:26:41.924 4076 3051113163944 MainThread DEBUG oslo_concurrency.processutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000084\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmprlxy9tue" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:26:42.301 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Setting boot order for instance: instance-00000084: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:26:42.362 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:26:42.408 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '3C62926C-720C-4A6F-B366-5E61083CC1BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18537), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:42.541 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:42.541 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:42.541 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:42.541 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:42.556 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3C62926C-720C-4A6F-B366-5E61083CC1BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114199), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:42.556 4076 3051113163944 MainThread DEBUG os_win._utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1938779354095459 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:26:42.556 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-6e0094f9-ae88-4713-b4d8-0f7212ba0d4f - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:42.556 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-6e0094f9-ae88-4713-b4d8-0f7212ba0d4f - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:42.598 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-6e0094f9-ae88-4713-b4d8-0f7212ba0d4f - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:42.633 4076 3051055217128 MainThread INFO nova.compute.manager [req-6e0094f9-ae88-4713-b4d8-0f7212ba0d4f - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:43.583 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '06C52DED-A4D1-4505-B8C0-D991461A508F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15547), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:43.705 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:43.705 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:43.705 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:43.705 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:43.705 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '06C52DED-A4D1-4505-B8C0-D991461A508F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112854), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:43.705 4076 3051113163944 MainThread DEBUG os_win._utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3426506519317627 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:26:43.733 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-598c0714-bea9-4509-a639-909e620746a6 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:43.733 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-598c0714-bea9-4509-a639-909e620746a6 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:43.775 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-598c0714-bea9-4509-a639-909e620746a6 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:43.789 4076 3051055217128 MainThread INFO nova.compute.manager [req-598c0714-bea9-4509-a639-909e620746a6 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:44.738 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '5E945571-F6B4-48D6-9B85-C0C2B1596FE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14928), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:44.879 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:44.879 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:44.879 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:44.879 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:44.879 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5E945571-F6B4-48D6-9B85-C0C2B1596FE9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110649), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:44.879 4076 3051113163944 MainThread DEBUG os_win._utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5168707370758057 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:26:44.879 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-3994419e-27db-413d-81bf-075dcf6e5cd5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:44.879 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-3994419e-27db-413d-81bf-075dcf6e5cd5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:44.943 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-3994419e-27db-413d-81bf-075dcf6e5cd5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:44.957 4076 3051055217128 MainThread INFO nova.compute.manager [req-3994419e-27db-413d-81bf-075dcf6e5cd5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:45.923 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '6884F568-F5D5-4B19-B8CD-762DAA4E5353', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15051), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:46.051 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:46.051 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:46.051 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:46.051 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:46.071 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6884F568-F5D5-4B19-B8CD-762DAA4E5353', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111296), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:46.071 4076 3051113163944 MainThread DEBUG os_win._utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7081477642059326 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:26:46.071 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d130d0a2-904b-46b1-b56e-c464d445223e - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:46.071 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-d130d0a2-904b-46b1-b56e-c464d445223e - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:46.113 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d130d0a2-904b-46b1-b56e-c464d445223e - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:46.145 4076 3051055217128 MainThread INFO nova.compute.manager [req-d130d0a2-904b-46b1-b56e-c464d445223e - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:47.097 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'D8A1D464-C00B-42F1-B7B8-AEE440F08468', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19710), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:47.221 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:47.221 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:47.221 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:47.221 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:47.252 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D8A1D464-C00B-42F1-B7B8-AEE440F08468', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113110), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:47.252 4076 3051113163944 MainThread DEBUG os_win._utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.889034271240234 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:26:47.252 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d661d51c-56eb-455f-8e1b-efd896b2b8c3 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:47.252 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-d661d51c-56eb-455f-8e1b-efd896b2b8c3 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:47.298 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d661d51c-56eb-455f-8e1b-efd896b2b8c3 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:47.316 4076 3051055217128 MainThread INFO nova.compute.manager [req-d661d51c-56eb-455f-8e1b-efd896b2b8c3 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:48.277 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': '15CE736A-377E-41DC-9C3C-FEE91096A8C4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15266), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:48.413 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:26:48.413 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] VM Stopped (Lifecycle Event) 2021-09-10 17:26:48.413 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:48.413 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:48.413 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000084\' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000084\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)instance-00000084CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '15CE736A-377E-41DC-9C3C-FEE91096A8C4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110818), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 26, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 26, 48), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\nNot enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)\n\n'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2)", 'ErrorSummaryDescription': "'instance-00000084' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000084.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:26:48.413 4076 3051113163944 MainThread ERROR nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Failed to change vm state of instance-00000084 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. 2021-09-10 17:26:48.436 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Got request to destroy instance 2021-09-10 17:26:48.436 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-06b215f2-da61-434e-98c1-8913515ae0b5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:26:48.436 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-06b215f2-da61-434e-98c1-8913515ae0b5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:26:48.472 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-06b215f2-da61-434e-98c1-8913515ae0b5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:26:48.493 4076 3051055217128 MainThread INFO nova.compute.manager [req-06b215f2-da61-434e-98c1-8913515ae0b5 - - - - -] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:26:50.246 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:26:50.246 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:50.246 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:50.252 4076 3051113163944 MainThread DEBUG os_win.utils.compute.vmutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed vm state of instance-00000084 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:26:50.252 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Successfully changed state of VM instance-00000084 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:26:50.301 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Waiting for WMI job: {'InstanceID': 'B8008461-8406-49CC-86ED-5444EF740090', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14695), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:26:50.442 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] WMI job succeeded: {'InstanceID': 'B8008461-8406-49CC-86ED-5444EF740090', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30781), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:26:50.442 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:50.442 4076 3051113163944 MainThread DEBUG os_win.utils.pathutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing directory: C:\openstack\instances\instance-00000084 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:26:50.442 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:50.442 4076 3051113163944 MainThread ERROR nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Traceback (most recent call last): 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(*args, **kwargs) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] admin_password, network_info, block_device_info) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(self, *args, **kwds) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] should_plug_vifs=False) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'req_state': req_state}) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] time.sleep(sleep_time) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return f(*args, **kwargs) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return self._wait_for_job(job_path) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] error_desc=err_desc) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During handling of the above exception, another exception occurred: 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Traceback (most recent call last): 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] yield resources 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] accel_info=accel_info) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(*args, **kwargs) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] admin_password, network_info, block_device_info) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(self, *args, **kwds) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] should_plug_vifs=False) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'req_state': req_state}) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] time.sleep(sleep_time) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return f(*args, **kwargs) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return self._wait_for_job(job_path) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] error_desc=err_desc) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:50.442 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:50.460 4076 3051113163944 MainThread INFO nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Terminating instance 2021-09-10 17:26:50.460 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:26:50.460 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Got request to destroy instance 2021-09-10 17:26:50.460 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:26:50.460 4076 3051113163944 MainThread DEBUG os_brick.initiator.windows.iscsi [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980', 'target_lun': 0, 'volume_id': '1379018d-523d-44fa-961c-1c45d19b8980', 'auth_method': 'CHAP', 'auth_username': 'ZhSmYdnG6t2N8fdpK8zj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:26:50.460 4076 3051113163944 MainThread DEBUG os_win.utils.storage.diskutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:26:50.567 4076 3051113163944 MainThread DEBUG os_win.utils.storage.diskutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:26:50.581 4076 3051113163944 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980 logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:26:50.648 4076 3051113163944 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-1379018d-523d-44fa-961c-1c45d19b8980 _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:26:50.648 4076 3051113163944 MainThread DEBUG os_brick.initiator.windows.iscsi [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] <== disconnect_volume: return (188ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:26:50.648 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:50.648 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "instance-00000084" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:50.648 4076 3051113163944 MainThread INFO nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Took 0.19 seconds to destroy the instance on the hypervisor. 2021-09-10 17:26:52.862 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:52.862 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:26:52.925 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:26:52.925 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:26:53.439 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-1ca070a8-791d-492e-9d9e-fd15b829fc9e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Received event network-vif-unplugged-674c16b2-c09e-42c8-9819-4931bce2b4da external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:26:53.439 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-1ca070a8-791d-492e-9d9e-fd15b829fc9e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:53.439 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-1ca070a8-791d-492e-9d9e-fd15b829fc9e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:53.439 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-1ca070a8-791d-492e-9d9e-fd15b829fc9e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] No waiting events found dispatching network-vif-unplugged-674c16b2-c09e-42c8-9819-4931bce2b4da pop_instance_event c:\openstack\build\nova\nova\compute\manager.py:319 2021-09-10 17:26:53.439 4076 3051113161304 MainThread WARNING nova.compute.manager [req-1ca070a8-791d-492e-9d9e-fd15b829fc9e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Received unexpected event network-vif-unplugged-674c16b2-c09e-42c8-9819-4931bce2b4da for instance with vm_state building and task_state spawning. 2021-09-10 17:26:53.455 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:26:53.482 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:26:53.482 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:26:53.892 4076 3051113163944 MainThread INFO nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Took 3.25 seconds to detach 1 volumes for instance. 2021-09-10 17:26:53.892 4076 3051113163944 MainThread DEBUG nova.compute.claims [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:26:53.892 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:26:54.017 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:26:54.021 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:26:54.221 4076 3051113163944 MainThread DEBUG nova.compute.provider_tree [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.scheduler.client.report [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:54.251 4076 3051113163944 MainThread ERROR nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Traceback (most recent call last): 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(*args, **kwargs) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] admin_password, network_info, block_device_info) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(self, *args, **kwds) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] should_plug_vifs=False) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'req_state': req_state}) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] time.sleep(sleep_time) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return f(*args, **kwargs) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return self._wait_for_job(job_path) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] error_desc=err_desc) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] During handling of the above exception, another exception occurred: 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Traceback (most recent call last): 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] accel_info=accel_info) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(*args, **kwargs) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] admin_password, network_info, block_device_info) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return function(self, *args, **kwds) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] should_plug_vifs=False) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'req_state': req_state}) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] time.sleep(sleep_time) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self.force_reraise() 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] raise self.value 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return f(*args, **kwargs) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] return self._wait_for_job(job_path) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] error_desc=err_desc) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. 2021-09-10 17:26:54.251 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.compute.utils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Build of instance 58aa12fc-aae6-43d2-8479-9e7a28651c3b was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000084' failed to start. Not enough memory in the system to start the virtual machine instance-00000084. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000084' failed to start. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Not enough memory in the system to start the virtual machine instance-00000084 with ram size 1024 megabytes. (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) 'instance-00000084' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID CE39CED9-EEA2-4E22-B832-2B98D1B7FBB2) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:26:54.251 4076 3051113163944 MainThread DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:26:54.738 4076 3051113163944 MainThread DEBUG nova.network.neutron [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:26:54.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:54.767 4076 3051113163944 MainThread INFO nova.compute.manager [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] [instance: 58aa12fc-aae6-43d2-8479-9e7a28651c3b] Took 0.52 seconds to deallocate network for instance. 2021-09-10 17:26:54.941 4076 3051113163944 MainThread INFO nova.scheduler.client.report [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Deleted allocations for instance 58aa12fc-aae6-43d2-8479-9e7a28651c3b 2021-09-10 17:26:54.972 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-ce724d14-5a80-4849-837d-f825354b7a92 82eacb807eb449a28e17290cd4d841d0 8d0aa6ac8af84d7291f244f216bb94b1 - default default] Lock "58aa12fc-aae6-43d2-8479-9e7a28651c3b" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 27.765s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:26:57.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:58.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:58.782 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:26:58.798 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:27:00.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:01.741 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:02.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:02.777 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:02.777 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:02.777 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:27:02.777 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:27:02.801 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:27:03.017 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2198MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:27:03.017 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:27:03.207 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:27:03.270 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:27:03.301 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:27:03.301 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:27:03.329 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:27:03.374 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:27:03.374 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:27:03.385 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:27:03.595 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:27:03.610 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:27:03.610 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:27:03.610 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.594s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:05.595 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:05.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:10.762 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:13.774 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:13.774 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:27:27.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:27.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:27:27.770 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:27:47.265 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:47.284 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:27:47.378 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:47.378 4076 3051113161832 MainThread DEBUG nova.virt.hardware [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:27:47.378 4076 3051113161832 MainThread INFO nova.compute.claims [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Claim successful on node n-h1-808474-1 2021-09-10 17:27:47.472 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.hostops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:27:47.505 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.hostops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:27:47.706 4076 3051113161832 MainThread DEBUG nova.compute.provider_tree [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:27:47.733 4076 3051113161832 MainThread DEBUG nova.scheduler.client.report [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:27:47.738 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.360s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:47.738 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:27:47.800 4076 3051113161832 MainThread DEBUG nova.compute.utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:27:47.800 4076 3051055217128 GreenThread-121 DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:27:47.800 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:27:47.800 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:27:47.925 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:27:47.925 4076 3051055217128 GreenThread-121 DEBUG nova.policy [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'b513993c7b2a49f58a5612ac6bf20582', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce8959bb67354f96b60bf89ed2bef450', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:27:47.953 4076 3051113161832 MainThread INFO nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Spawning new instance 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG nova.block_device [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG os_win.utils.pathutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Creating directory: C:\openstack\instances\instance-00000085 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:27:47.956 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000085\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:27:47.991 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Resizing VHD C:\openstack\instances\instance-00000085\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:27:48.648 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Successfully created port: 5791c793-10cf-4885-868a-83878ab93fe6 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:27:49.303 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Successfully updated port: 5791c793-10cf-4885-868a-83878ab93fe6 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:27:49.331 4076 3051055217128 GreenThread-121 DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Acquired lock "refresh_cache-3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:27:49.331 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:27:49.359 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Received event network-changed-5791c793-10cf-4885-868a-83878ab93fe6 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:27:49.359 4076 3051113163768 MainThread DEBUG nova.compute.manager [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Refreshing instance network info cache due to event network-changed-5791c793-10cf-4885-868a-83878ab93fe6. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:27:49.424 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:27:49.765 4076 3051055217128 GreenThread-121 DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Updating instance_info_cache with network_info: [{"id": "5791c793-10cf-4885-868a-83878ab93fe6", "address": "fa:16:3e:de:c7:54", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.170", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5791c793-10", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:27:49.781 4076 3051055217128 GreenThread-121 DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Releasing lock "refresh_cache-3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:27:49.781 4076 3051055217128 GreenThread-121 DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Instance network_info: |[{"id": "5791c793-10cf-4885-868a-83878ab93fe6", "address": "fa:16:3e:de:c7:54", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.170", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5791c793-10", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:27:49.781 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:27:49.781 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Refreshing network info cache for port 5791c793-10cf-4885-868a-83878ab93fe6 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:27:49.781 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Preparing to wait for external event network-vif-plugged-5791c793-10cf-4885-868a-83878ab93fe6 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:27:49.781 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:49.781 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:49.781 4076 3051113161832 MainThread DEBUG os_win.utils.compute.vmutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Creating VM instance-00000085 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:27:50.272 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Updated VIF entry in instance network info cache for port 5791c793-10cf-4885-868a-83878ab93fe6. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:27:50.272 4076 3051113163768 MainThread DEBUG nova.network.neutron [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Updating instance_info_cache with network_info: [{"id": "5791c793-10cf-4885-868a-83878ab93fe6", "address": "fa:16:3e:de:c7:54", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.170", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5791c793-10", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:27:50.287 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [req-9bb9f57f-587b-43fa-b9ce-6dd885d601b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:27:50.680 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:27:53.765 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:53.765 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:27:53.838 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:27:53.838 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:27:54.612 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:27:54.638 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:27:54.638 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:27:54.781 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:55.815 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-999bc899-4f9a-4e3d-b6e7-46bbe6646d68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Received event network-vif-plugged-5791c793-10cf-4885-868a-83878ab93fe6 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:27:55.815 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-999bc899-4f9a-4e3d-b6e7-46bbe6646d68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:55.815 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-999bc899-4f9a-4e3d-b6e7-46bbe6646d68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:55.815 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-999bc899-4f9a-4e3d-b6e7-46bbe6646d68 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Processing event network-vif-plugged-5791c793-10cf-4885-868a-83878ab93fe6 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:27:55.843 4076 3051113161832 MainThread INFO nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Using config drive for instance 2021-09-10 17:27:55.940 4076 3051113161832 MainThread INFO nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Creating config drive at C:\openstack\instances\instance-00000085\configdrive.iso 2021-09-10 17:27:55.987 4076 3051113161832 MainThread DEBUG oslo_concurrency.processutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000085\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpc3hamocf execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:27:56.283 4076 3051113161832 MainThread DEBUG oslo_concurrency.processutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000085\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpc3hamocf" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:27:56.685 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Setting boot order for instance: instance-00000085: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:27:56.734 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:27:56.768 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': '137E20BC-A871-4EBA-90D1-0A8BCC2C4563', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19587), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:27:56.893 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:27:56.893 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:27:56.893 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:56.893 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:56.922 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '137E20BC-A871-4EBA-90D1-0A8BCC2C4563', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111360), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 27, 56), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 27, 56), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:27:56.922 4076 3051113161832 MainThread DEBUG os_win._utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1872851848602295 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:27:56.922 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-46fc65a4-7e72-479b-8ab7-df4bb4e28f48 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:27:56.922 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-46fc65a4-7e72-479b-8ab7-df4bb4e28f48 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:27:56.956 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-46fc65a4-7e72-479b-8ab7-df4bb4e28f48 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:27:56.984 4076 3051055217128 MainThread INFO nova.compute.manager [req-46fc65a4-7e72-479b-8ab7-df4bb4e28f48 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:27:57.940 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': 'A49CC6E7-E906-4B1B-988D-FDFE65998875', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15595), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:27:58.089 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:27:58.089 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:27:58.089 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:58.089 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:58.089 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A49CC6E7-E906-4B1B-988D-FDFE65998875', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115088), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 27, 57), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 27, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:27:58.097 4076 3051113161832 MainThread DEBUG os_win._utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.362267017364502 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:27:58.097 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-86350c79-406f-41c9-8c1e-792f9bf6d754 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:27:58.097 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-86350c79-406f-41c9-8c1e-792f9bf6d754 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:27:58.143 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-86350c79-406f-41c9-8c1e-792f9bf6d754 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:27:58.172 4076 3051055217128 MainThread INFO nova.compute.manager [req-86350c79-406f-41c9-8c1e-792f9bf6d754 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:27:59.125 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': 'D91A3D66-79A9-471D-8289-DE8C25797373', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14451), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:27:59.256 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:27:59.256 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:27:59.256 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:27:59.256 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:27:59.268 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D91A3D66-79A9-471D-8289-DE8C25797373', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113489), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 27, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 27, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:27:59.268 4076 3051113161832 MainThread DEBUG os_win._utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5340917110443115 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:27:59.284 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-0e2737de-b954-485a-82ff-4c05f84d4ee8 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:27:59.284 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-0e2737de-b954-485a-82ff-4c05f84d4ee8 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:27:59.330 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-0e2737de-b954-485a-82ff-4c05f84d4ee8 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:27:59.362 4076 3051055217128 MainThread INFO nova.compute.manager [req-0e2737de-b954-485a-82ff-4c05f84d4ee8 - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:27:59.765 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:59.765 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:27:59.765 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:27:59.968 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:00.284 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': 'A0E04274-5B60-46B5-88AF-CFEC3F010AEF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15870), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:00.422 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A0E04274-5B60-46B5-88AF-CFEC3F010AEF', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109968), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:00.422 4076 3051113161832 MainThread DEBUG os_win._utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.687237501144409 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:00.436 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:00.436 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:00.436 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:00.436 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:28:00.456 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-3e3b74c2-f29a-4ce3-8f85-7eca75faddeb - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:00.456 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-3e3b74c2-f29a-4ce3-8f85-7eca75faddeb - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:00.505 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-3e3b74c2-f29a-4ce3-8f85-7eca75faddeb - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:00.518 4076 3051055217128 MainThread INFO nova.compute.manager [req-3e3b74c2-f29a-4ce3-8f85-7eca75faddeb - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:00.796 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:01.445 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': 'B794DADC-4D90-45DC-A859-F8F1A8B2587D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14654), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:01.581 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:01.581 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:01.581 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:01.581 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:28:01.607 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-4515b3d2-c877-4e92-bb42-805b2e31752d - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:01.611 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-4515b3d2-c877-4e92-bb42-805b2e31752d - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:01.611 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B794DADC-4D90-45DC-A859-F8F1A8B2587D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112750), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:01.611 4076 3051113161832 MainThread DEBUG os_win._utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.876621723175049 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:01.648 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-4515b3d2-c877-4e92-bb42-805b2e31752d - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:01.672 4076 3051055217128 MainThread INFO nova.compute.manager [req-4515b3d2-c877-4e92-bb42-805b2e31752d - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:02.625 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': '97FCCC01-9ED8-4DF9-81CC-CEECF898D773', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15978), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:02.759 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:02.759 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:02.759 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:02.759 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:02.759 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] VM Stopped (Lifecycle Event) 2021-09-10 17:28:02.781 4076 3051113161832 MainThread ERROR os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000085\' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF26210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000085\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)instance-0000008560822D8C-F937-46A3-8AB2-55CC0D2EF262%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '97FCCC01-9ED8-4DF9-81CC-CEECF898D773', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111233), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\nNot enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)\n\n'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262)", 'ErrorSummaryDescription': "'instance-00000085' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000085.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:02.781 4076 3051113161832 MainThread ERROR nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Failed to change vm state of instance-00000085 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. 2021-09-10 17:28:02.781 4076 3051113161832 MainThread INFO nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Got request to destroy instance 2021-09-10 17:28:02.781 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-9286974f-3bee-4f35-924f-b4381b888e6c - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:02.781 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-9286974f-3bee-4f35-924f-b4381b888e6c - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:02.836 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-9286974f-3bee-4f35-924f-b4381b888e6c - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:02.864 4076 3051055217128 MainThread INFO nova.compute.manager [req-9286974f-3bee-4f35-924f-b4381b888e6c - - - - -] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:03.734 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:04.579 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:28:04.579 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:04.579 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:04.604 4076 3051113161832 MainThread DEBUG os_win.utils.compute.vmutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Successfully changed vm state of instance-00000085 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:28:04.604 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Successfully changed state of VM instance-00000085 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:28:04.663 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Waiting for WMI job: {'InstanceID': 'FBCDA64C-8DD1-4A00-8E4E-2A0D9AF3E064', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14249), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:04.776 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:04.784 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:04.784 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:04.784 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:28:04.784 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:28:04.784 4076 3051113161832 MainThread DEBUG os_win.utils.jobutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] WMI job succeeded: {'InstanceID': 'FBCDA64C-8DD1-4A00-8E4E-2A0D9AF3E064', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28421), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:28:04.784 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:04.784 4076 3051113161832 MainThread DEBUG os_win.utils.pathutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Removing directory: C:\openstack\instances\instance-00000085 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:28:04.784 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:04.784 4076 3051113161832 MainThread ERROR nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Traceback (most recent call last): 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(*args, **kwargs) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] admin_password, network_info, block_device_info) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(self, *args, **kwds) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] should_plug_vifs=False) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'req_state': req_state}) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] time.sleep(sleep_time) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return f(*args, **kwargs) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return self._wait_for_job(job_path) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] error_desc=err_desc) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During handling of the above exception, another exception occurred: 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Traceback (most recent call last): 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] yield resources 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] accel_info=accel_info) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(*args, **kwargs) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] admin_password, network_info, block_device_info) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(self, *args, **kwds) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] should_plug_vifs=False) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'req_state': req_state}) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] time.sleep(sleep_time) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return f(*args, **kwargs) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return self._wait_for_job(job_path) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] error_desc=err_desc) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:04.784 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:04.784 4076 3051113161832 MainThread INFO nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Terminating instance 2021-09-10 17:28:04.784 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:28:04.784 4076 3051113161832 MainThread INFO nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Got request to destroy instance 2021-09-10 17:28:04.815 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:28:04.815 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:28:04.815 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:04.815 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "instance-00000085" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:04.815 4076 3051113161832 MainThread INFO nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:28:04.830 4076 3051113161832 MainThread DEBUG nova.compute.claims [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:28:04.830 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:04.903 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.hostops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:28:04.930 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.hostops [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:28:05.032 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2198MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:28:05.140 4076 3051113161832 MainThread DEBUG nova.compute.provider_tree [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.scheduler.client.report [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:05.156 4076 3051113161832 MainThread ERROR nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Traceback (most recent call last): 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(*args, **kwargs) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] admin_password, network_info, block_device_info) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(self, *args, **kwds) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] should_plug_vifs=False) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'req_state': req_state}) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] time.sleep(sleep_time) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return f(*args, **kwargs) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return self._wait_for_job(job_path) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] error_desc=err_desc) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] During handling of the above exception, another exception occurred: 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Traceback (most recent call last): 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] accel_info=accel_info) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(*args, **kwargs) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] admin_password, network_info, block_device_info) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return function(self, *args, **kwds) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] should_plug_vifs=False) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'req_state': req_state}) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] time.sleep(sleep_time) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self.force_reraise() 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] raise self.value 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return f(*args, **kwargs) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] return self._wait_for_job(job_path) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] error_desc=err_desc) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. 2021-09-10 17:28:05.156 4076 3051113161832 MainThread TRACE nova.compute.manager [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.compute.utils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Build of instance 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000085' failed to start. Not enough memory in the system to start the virtual machine instance-00000085. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000085' failed to start. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Not enough memory in the system to start the virtual machine instance-00000085 with ram size 1024 megabytes. (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) 'instance-00000085' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 60822D8C-F937-46A3-8AB2-55CC0D2EF262) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:28:05.156 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:28:05.156 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.283 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:28:05.312 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1591 2021-09-10 17:28:05.312 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:28:05.312 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:28:05.343 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:28:05.363 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:28:05.565 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:28:05.578 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:28:05.578 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:28:05.578 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:05.674 4076 3051113161832 MainThread DEBUG nova.network.neutron [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:28:05.703 4076 3051113161832 MainThread INFO nova.compute.manager [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] [instance: 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f] Took 0.55 seconds to deallocate network for instance. 2021-09-10 17:28:05.862 4076 3051113161832 MainThread INFO nova.scheduler.client.report [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Deleted allocations for instance 3568f444-8a4d-4c97-8bb4-ef66ca1eae4f 2021-09-10 17:28:05.893 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [req-dfdc8377-fe21-48ed-b3cc-906ef23770d4 b513993c7b2a49f58a5612ac6bf20582 ce8959bb67354f96b60bf89ed2bef450 - default default] Lock "3568f444-8a4d-4c97-8bb4-ef66ca1eae4f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.625s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:11.566 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:34.189 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:34.210 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:28:34.299 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:34.299 4076 3051113162184 MainThread DEBUG nova.virt.hardware [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:28:34.299 4076 3051113162184 MainThread INFO nova.compute.claims [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Claim successful on node n-h1-808474-1 2021-09-10 17:28:34.405 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:28:34.424 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:28:34.619 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:28:34.635 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:28:34.635 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:34.635 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:28:34.706 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:28:34.706 4076 3051055217128 GreenThread-122 DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:28:34.706 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:28:34.721 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:28:34.827 4076 3051055217128 GreenThread-122 DEBUG nova.policy [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '49db170bc6d5400aa9ebfb5a82a74ca6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5df26eb38b4042e58ed51a4332022a9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:28:34.827 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:28:34.827 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Spawning new instance 2021-09-10 17:28:34.856 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:34.856 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:34.856 4076 3051113162184 MainThread DEBUG nova.block_device [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:28:34.856 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:34.856 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:34.872 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Creating directory: C:\openstack\instances\instance-00000086 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:28:34.872 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000086\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:28:34.890 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Resizing VHD C:\openstack\instances\instance-00000086\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:28:35.398 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Successfully created port: ad889853-c275-4154-8dfe-74957267b329 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:28:35.952 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:35.954 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Successfully updated port: ad889853-c275-4154-8dfe-74957267b329 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:28:36.006 4076 3051055217128 GreenThread-122 DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Acquired lock "refresh_cache-2593bf32-6e0d-4942-9e43-2e781b198fab" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:28:36.006 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:28:36.006 4076 3051049642296 GreenThread-2 WARNING nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] While synchronizing instance power states, found 13 instances in the database and 12 instances on the hypervisor. 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2593bf32-6e0d-4942-9e43-2e781b198fab _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:28:36.031 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113163768 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113161304 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113162536 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113161832 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.031 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.031 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:36.049 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Received event network-changed-ad889853-c275-4154-8dfe-74957267b329 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:28:36.049 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Refreshing instance network info cache due to event network-changed-ad889853-c275-4154-8dfe-74957267b329. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:28:36.093 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:28:36.188 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.188 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.188 4076 3051113163768 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.188 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.214 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.214 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.219 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.502 4076 3051055217128 GreenThread-122 DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Updating instance_info_cache with network_info: [{"id": "ad889853-c275-4154-8dfe-74957267b329", "address": "fa:16:3e:cf:76:ea", "network": {"id": "8d49abc1-73c3-44ea-a0ca-c19705eb8ff4", "bridge": null, "label": "tempest-DeleteServersTestJSON-1955226753-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5df26eb38b4042e58ed51a4332022a9c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapad889853-c2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:28:36.533 4076 3051055217128 GreenThread-122 DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Releasing lock "refresh_cache-2593bf32-6e0d-4942-9e43-2e781b198fab" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:28:36.533 4076 3051055217128 GreenThread-122 DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Instance network_info: |[{"id": "ad889853-c275-4154-8dfe-74957267b329", "address": "fa:16:3e:cf:76:ea", "network": {"id": "8d49abc1-73c3-44ea-a0ca-c19705eb8ff4", "bridge": null, "label": "tempest-DeleteServersTestJSON-1955226753-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5df26eb38b4042e58ed51a4332022a9c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapad889853-c2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:28:36.533 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-2593bf32-6e0d-4942-9e43-2e781b198fab" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:28:36.533 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Refreshing network info cache for port ad889853-c275-4154-8dfe-74957267b329 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:28:36.533 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Preparing to wait for external event network-vif-plugged-ad889853-c275-4154-8dfe-74957267b329 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:28:36.533 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:36.533 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:36.533 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Creating VM instance-00000086 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:28:37.002 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Updated VIF entry in instance network info cache for port ad889853-c275-4154-8dfe-74957267b329. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:28:37.002 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Updating instance_info_cache with network_info: [{"id": "ad889853-c275-4154-8dfe-74957267b329", "address": "fa:16:3e:cf:76:ea", "network": {"id": "8d49abc1-73c3-44ea-a0ca-c19705eb8ff4", "bridge": null, "label": "tempest-DeleteServersTestJSON-1955226753-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5df26eb38b4042e58ed51a4332022a9c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapad889853-c2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:28:37.017 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [req-05b30182-01de-40f8-8408-6c849fb1ca52 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-2593bf32-6e0d-4942-9e43-2e781b198fab" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:28:37.406 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:28:42.077 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-ef685374-8fdb-49ef-a3bb-2fd9ae8ffe64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Received event network-vif-plugged-ad889853-c275-4154-8dfe-74957267b329 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:28:42.077 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-ef685374-8fdb-49ef-a3bb-2fd9ae8ffe64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:42.077 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-ef685374-8fdb-49ef-a3bb-2fd9ae8ffe64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:42.077 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-ef685374-8fdb-49ef-a3bb-2fd9ae8ffe64 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Processing event network-vif-plugged-ad889853-c275-4154-8dfe-74957267b329 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:28:42.095 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Using config drive for instance 2021-09-10 17:28:42.236 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Creating config drive at C:\openstack\instances\instance-00000086\configdrive.iso 2021-09-10 17:28:42.283 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000086\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyfpjhsji execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:28:42.597 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000086\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyfpjhsji" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:28:42.985 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Setting boot order for instance: instance-00000086: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:28:43.036 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:28:43.064 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': 'D440FDE6-B73C-4D58-B9F2-35293479213B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15226), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:43.205 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:43.205 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:43.205 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:43.205 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:43.205 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D440FDE6-B73C-4D58-B9F2-35293479213B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113873), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:43.205 4076 3051113162184 MainThread DEBUG os_win._utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16898345947265625 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:43.220 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-c6b56c44-3e97-4751-af90-04845f13a99e - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:43.220 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-c6b56c44-3e97-4751-af90-04845f13a99e - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:43.269 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-c6b56c44-3e97-4751-af90-04845f13a99e - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:43.290 4076 3051055217128 MainThread INFO nova.compute.manager [req-c6b56c44-3e97-4751-af90-04845f13a99e - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:44.221 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': '38E4F63D-B762-4974-89F0-6EE8FF3E25C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14968), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:44.360 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:44.360 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:44.360 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:44.360 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:44.360 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '38E4F63D-B762-4974-89F0-6EE8FF3E25C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110049), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:44.360 4076 3051113162184 MainThread DEBUG os_win._utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3239562511444092 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:44.386 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d8262fce-54a5-4c49-8b18-1ab15533a3a6 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:44.386 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-d8262fce-54a5-4c49-8b18-1ab15533a3a6 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:44.428 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d8262fce-54a5-4c49-8b18-1ab15533a3a6 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:44.455 4076 3051055217128 MainThread INFO nova.compute.manager [req-d8262fce-54a5-4c49-8b18-1ab15533a3a6 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:45.383 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': '4EE01A63-0693-4944-9521-0E8F6D83E54C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15680), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:45.508 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:45.525 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:45.525 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:45.525 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:45.530 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4EE01A63-0693-4944-9521-0E8F6D83E54C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110755), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 45), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:45.530 4076 3051113162184 MainThread DEBUG os_win._utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.4937584400177 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:45.530 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-41baf039-80ba-46c2-8185-5f2fe7746c8b - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:45.530 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-41baf039-80ba-46c2-8185-5f2fe7746c8b - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:45.596 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-41baf039-80ba-46c2-8185-5f2fe7746c8b - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:45.611 4076 3051055217128 MainThread INFO nova.compute.manager [req-41baf039-80ba-46c2-8185-5f2fe7746c8b - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:46.568 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': '073941E7-79D6-4A26-A639-5853532E2551', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15249), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:46.686 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:46.686 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:46.686 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:46.686 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:46.714 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '073941E7-79D6-4A26-A639-5853532E2551', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109519), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 46), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:46.714 4076 3051113162184 MainThread DEBUG os_win._utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6780247688293457 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:46.714 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d9fc56c2-9311-467c-b7cd-3506f8a530b5 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:46.733 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-d9fc56c2-9311-467c-b7cd-3506f8a530b5 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:46.767 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-d9fc56c2-9311-467c-b7cd-3506f8a530b5 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:46.797 4076 3051055217128 MainThread INFO nova.compute.manager [req-d9fc56c2-9311-467c-b7cd-3506f8a530b5 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:47.752 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': '6D154A43-4C53-4B98-82EC-DA486E882F24', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15380), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:47.883 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:47.883 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:47.883 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:47.883 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:47.896 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6D154A43-4C53-4B98-82EC-DA486E882F24', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117291), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:47.896 4076 3051113162184 MainThread DEBUG os_win._utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8601460456848145 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:28:47.896 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8c5a29df-701c-4653-87c0-39912a06ab83 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:47.908 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8c5a29df-701c-4653-87c0-39912a06ab83 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:47.953 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8c5a29df-701c-4653-87c0-39912a06ab83 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:47.953 4076 3051055217128 MainThread INFO nova.compute.manager [req-8c5a29df-701c-4653-87c0-39912a06ab83 - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:48.923 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': '9C5B4FF3-2A78-424E-9A97-B3CAEEA3F1AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14806), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:49.054 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:28:49.054 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] VM Stopped (Lifecycle Event) 2021-09-10 17:28:49.054 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:49.054 4076 3051113161832 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:49.077 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8987e6bb-c99b-44c2-9fda-6d77ef427bfb - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:28:49.077 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-8987e6bb-c99b-44c2-9fda-6d77ef427bfb - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:28:49.077 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job failed: {'RawErrors': (('124\'instance-00000086\' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000086\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)instance-00000086B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9C5B4FF3-2A78-424E-9A97-B3CAEEA3F1AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110274), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 28, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 28, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\nNot enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)\n\n'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9)", 'ErrorSummaryDescription': "'instance-00000086' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000086.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:28:49.077 4076 3051113162184 MainThread ERROR nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Failed to change vm state of instance-00000086 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. 2021-09-10 17:28:49.077 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Got request to destroy instance 2021-09-10 17:28:49.110 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-8987e6bb-c99b-44c2-9fda-6d77ef427bfb - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:28:49.148 4076 3051055217128 MainThread INFO nova.compute.manager [req-8987e6bb-c99b-44c2-9fda-6d77ef427bfb - - - - -] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:50.877 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:28:50.877 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:50.877 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:50.905 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Successfully changed vm state of instance-00000086 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:28:50.905 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Successfully changed state of VM instance-00000086 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:28:50.955 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Waiting for WMI job: {'InstanceID': 'AAE6B620-E3DF-4207-938B-D333A9E7B9FD', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14455), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:28:51.064 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] WMI job succeeded: {'InstanceID': 'AAE6B620-E3DF-4207-938B-D333A9E7B9FD', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30423), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:28:51.064 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Removing directory: C:\openstack\instances\instance-00000086 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:51.080 4076 3051113162184 MainThread ERROR nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Traceback (most recent call last): 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(*args, **kwargs) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] admin_password, network_info, block_device_info) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(self, *args, **kwds) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] should_plug_vifs=False) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'req_state': req_state}) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] time.sleep(sleep_time) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return f(*args, **kwargs) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return self._wait_for_job(job_path) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] error_desc=err_desc) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During handling of the above exception, another exception occurred: 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Traceback (most recent call last): 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] yield resources 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] accel_info=accel_info) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(*args, **kwargs) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] admin_password, network_info, block_device_info) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(self, *args, **kwds) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] should_plug_vifs=False) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'req_state': req_state}) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] time.sleep(sleep_time) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return f(*args, **kwargs) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return self._wait_for_job(job_path) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] error_desc=err_desc) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.080 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.080 4076 3051113162184 MainThread INFO nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Terminating instance 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:28:51.080 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Got request to destroy instance 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:51.080 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "instance-00000086" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:51.080 4076 3051113162184 MainThread INFO nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:28:51.108 4076 3051113162184 MainThread DEBUG nova.compute.claims [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:28:51.108 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:51.181 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:28:51.202 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:28:51.412 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:28:51.412 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:28:51.412 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:51.412 4076 3051113162184 MainThread ERROR nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Traceback (most recent call last): 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(*args, **kwargs) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] admin_password, network_info, block_device_info) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(self, *args, **kwds) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] should_plug_vifs=False) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'req_state': req_state}) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] time.sleep(sleep_time) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return f(*args, **kwargs) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return self._wait_for_job(job_path) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] error_desc=err_desc) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During handling of the above exception, another exception occurred: 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Traceback (most recent call last): 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] accel_info=accel_info) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(*args, **kwargs) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] admin_password, network_info, block_device_info) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return function(self, *args, **kwds) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] should_plug_vifs=False) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'req_state': req_state}) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] time.sleep(sleep_time) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self.force_reraise() 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] raise self.value 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return f(*args, **kwargs) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] return self._wait_for_job(job_path) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] error_desc=err_desc) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. 2021-09-10 17:28:51.412 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Build of instance 2593bf32-6e0d-4942-9e43-2e781b198fab was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000086' failed to start. Not enough memory in the system to start the virtual machine instance-00000086. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000086' failed to start. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Not enough memory in the system to start the virtual machine instance-00000086 with ram size 1024 megabytes. (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) 'instance-00000086' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID B5B93B06-670B-4BCC-B2E5-7CAE0FFB92F9) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:28:51.431 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:28:52.182 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:28:52.202 4076 3051113162184 MainThread INFO nova.compute.manager [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] Took 0.77 seconds to deallocate network for instance. 2021-09-10 17:28:52.364 4076 3051113162184 MainThread INFO nova.scheduler.client.report [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Deleted allocations for instance 2593bf32-6e0d-4942-9e43-2e781b198fab 2021-09-10 17:28:52.398 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-681252e5-461f-4e4e-960b-9c29547c8091 49db170bc6d5400aa9ebfb5a82a74ca6 5df26eb38b4042e58ed51a4332022a9c - default default] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:52.398 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 16.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:28:52.398 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 2593bf32-6e0d-4942-9e43-2e781b198fab] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:28:52.398 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2593bf32-6e0d-4942-9e43-2e781b198fab" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:28:55.842 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:28:55.843 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:28:55.907 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:28:55.907 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:28:56.439 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:28:56.461 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:28:56.461 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:28:56.759 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:00.780 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:00.780 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:29:01.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:01.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:04.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:04.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:05.764 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:05.770 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:05.782 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:05.782 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:29:05.782 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:29:05.782 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:29:06.025 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2200MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:29:06.025 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:06.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:29:06.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:29:06.189 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:29:06.202 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:29:06.405 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:29:06.405 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:29:06.405 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:29:06.405 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:10.383 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:10.779 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:14.235 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:14.279 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:29:14.373 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:14.373 4076 3051113162360 MainThread DEBUG nova.virt.hardware [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:29:14.373 4076 3051113162360 MainThread INFO nova.compute.claims [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Claim successful on node n-h1-808474-1 2021-09-10 17:29:14.470 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:29:14.494 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:29:14.704 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:29:14.718 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:29:14.718 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:14.718 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:29:14.767 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:29:14.767 4076 3051113163240 GreenThread-123 DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:29:14.767 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:29:14.795 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:29:14.891 4076 3051113163240 GreenThread-123 DEBUG nova.policy [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '652641e6ad894a5cbe0cbe09bd677e6f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ece1a15a5d64ee19aacaadbaf9c9c95', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:29:14.908 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:29:14.930 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Spawning new instance 2021-09-10 17:29:14.951 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:14.951 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:14.951 4076 3051113162360 MainThread DEBUG nova.block_device [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:29:14.951 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:14.951 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:14.962 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Creating directory: C:\openstack\instances\instance-00000087 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:29:14.962 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000087\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:29:14.983 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Resizing VHD C:\openstack\instances\instance-00000087\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:29:15.454 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Successfully created port: e383628a-cded-46c3-a950-edaa6001088c _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:29:16.054 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Successfully updated port: e383628a-cded-46c3-a950-edaa6001088c _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:29:16.071 4076 3051113163240 GreenThread-123 DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Acquired lock "refresh_cache-ff096eb1-eabf-47db-b43c-94d9879eba49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:29:16.071 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:29:16.092 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Received event network-changed-e383628a-cded-46c3-a950-edaa6001088c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:29:16.092 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Refreshing instance network info cache due to event network-changed-e383628a-cded-46c3-a950-edaa6001088c. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:29:16.154 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:29:16.507 4076 3051113163240 GreenThread-123 DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Updating instance_info_cache with network_info: [{"id": "e383628a-cded-46c3-a950-edaa6001088c", "address": "fa:16:3e:b7:c7:74", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape383628a-cd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:16.515 4076 3051113163240 GreenThread-123 DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Releasing lock "refresh_cache-ff096eb1-eabf-47db-b43c-94d9879eba49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:29:16.515 4076 3051113163240 GreenThread-123 DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Instance network_info: |[{"id": "e383628a-cded-46c3-a950-edaa6001088c", "address": "fa:16:3e:b7:c7:74", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape383628a-cd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:29:16.515 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ff096eb1-eabf-47db-b43c-94d9879eba49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:29:16.515 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Refreshing network info cache for port e383628a-cded-46c3-a950-edaa6001088c _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:29:16.532 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Preparing to wait for external event network-vif-plugged-e383628a-cded-46c3-a950-edaa6001088c prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:29:16.532 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:16.532 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:16.532 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Creating VM instance-00000087 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:29:16.990 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Updated VIF entry in instance network info cache for port e383628a-cded-46c3-a950-edaa6001088c. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:29:16.990 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Updating instance_info_cache with network_info: [{"id": "e383628a-cded-46c3-a950-edaa6001088c", "address": "fa:16:3e:b7:c7:74", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape383628a-cd", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:16.990 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-5fd968fe-bfb7-4788-aaba-bebf024001fb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ff096eb1-eabf-47db-b43c-94d9879eba49" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:29:17.407 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:29:22.326 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-51ad8ce9-6892-4dda-8cc3-14579936bf37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Received event network-vif-plugged-e383628a-cded-46c3-a950-edaa6001088c external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:29:22.326 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-51ad8ce9-6892-4dda-8cc3-14579936bf37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:22.326 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-51ad8ce9-6892-4dda-8cc3-14579936bf37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:22.326 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-51ad8ce9-6892-4dda-8cc3-14579936bf37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Processing event network-vif-plugged-e383628a-cded-46c3-a950-edaa6001088c _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:29:22.357 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Using config drive for instance 2021-09-10 17:29:22.454 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Creating config drive at C:\openstack\instances\instance-00000087\configdrive.iso 2021-09-10 17:29:22.498 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000087\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmgxcgcvk execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:29:22.796 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000087\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpmgxcgcvk" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:29:23.178 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Setting boot order for instance: instance-00000087: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:29:23.222 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:29:23.253 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '06AEA6A9-436F-42EA-80BE-4451B6F885AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19404), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:23.376 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:23.376 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:23.376 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:23.376 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:23.376 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '06AEA6A9-436F-42EA-80BE-4451B6F885AC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113645), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:23.376 4076 3051113162360 MainThread DEBUG os_win._utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.15391850471496582 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:29:23.407 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-2d1fb39d-5f6a-46fa-bbd1-ac8d8759ac50 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:23.407 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-2d1fb39d-5f6a-46fa-bbd1-ac8d8759ac50 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:23.454 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-2d1fb39d-5f6a-46fa-bbd1-ac8d8759ac50 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:23.486 4076 3051113164120 MainThread INFO nova.compute.manager [req-2d1fb39d-5f6a-46fa-bbd1-ac8d8759ac50 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:24.407 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '0793BC27-305C-46E4-A9EB-4AF175B8A559', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15404), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:24.545 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:24.545 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:24.545 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:24.545 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:24.545 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0793BC27-305C-46E4-A9EB-4AF175B8A559', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113240), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:24.545 4076 3051113162360 MainThread DEBUG os_win._utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3227035999298096 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:29:24.567 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-cfc75da7-209a-4c26-b0f3-bfda6c1493f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:24.567 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-cfc75da7-209a-4c26-b0f3-bfda6c1493f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:24.609 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-cfc75da7-209a-4c26-b0f3-bfda6c1493f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:24.638 4076 3051113164120 MainThread INFO nova.compute.manager [req-cfc75da7-209a-4c26-b0f3-bfda6c1493f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:25.563 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '2FD1D81F-38DE-4B66-A679-39580176FADA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14564), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:25.702 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:25.702 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:25.702 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:25.702 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:25.702 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2FD1D81F-38DE-4B66-A679-39580176FADA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112785), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:25.702 4076 3051113162360 MainThread DEBUG os_win._utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.4796974658966064 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:29:25.719 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-abdf3686-8a67-4890-bcea-2062eb9da6de - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:25.719 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-abdf3686-8a67-4890-bcea-2062eb9da6de - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:25.765 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-abdf3686-8a67-4890-bcea-2062eb9da6de - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:25.782 4076 3051113164120 MainThread INFO nova.compute.manager [req-abdf3686-8a67-4890-bcea-2062eb9da6de - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:26.704 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '8B22123B-6C7A-4CD3-984F-551F5C7895CC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14117), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:26.860 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:26.860 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:26.860 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:26.860 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:26.888 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-ce9419ea-cf47-482c-800b-505853a70e98 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:26.888 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-ce9419ea-cf47-482c-800b-505853a70e98 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:26.935 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-ce9419ea-cf47-482c-800b-505853a70e98 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:26.953 4076 3051113164120 MainThread INFO nova.compute.manager [req-ce9419ea-cf47-482c-800b-505853a70e98 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:26.984 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8B22123B-6C7A-4CD3-984F-551F5C7895CC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110278), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:26.984 4076 3051113162360 MainThread DEBUG os_win._utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7621912956237793 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:29:28.006 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': 'D8940CE8-8C34-4E1A-8AF7-7F0592CFCC60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15408), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:28.146 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:28.146 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:28.146 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:28.146 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:28.146 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D8940CE8-8C34-4E1A-8AF7-7F0592CFCC60', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110734), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 28), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:28.146 4076 3051113162360 MainThread DEBUG os_win._utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.924010992050171 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:29:28.170 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-a8a2fd90-311d-45d5-a917-70140b7812c0 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:28.170 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-a8a2fd90-311d-45d5-a917-70140b7812c0 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:28.207 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-a8a2fd90-311d-45d5-a917-70140b7812c0 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:28.237 4076 3051113164120 MainThread INFO nova.compute.manager [req-a8a2fd90-311d-45d5-a917-70140b7812c0 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:29.172 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '0A69DD91-B73F-4E0D-A9FC-BB2B80B8E1F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14921), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:29.296 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:29.296 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:29.296 4076 3051113164120 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:29:29.296 4076 3051113164120 MainThread INFO nova.compute.manager [-] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] VM Stopped (Lifecycle Event) 2021-09-10 17:29:29.296 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000087\' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000087\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)instance-000000875DB5361A-2F61-4DDB-9E6B-9856B09AD78F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0A69DD91-B73F-4E0D-A9FC-BB2B80B8E1F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111616), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 29, 29), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 29, 29), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\nNot enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)\n\n'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F)", 'ErrorSummaryDescription': "'instance-00000087' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000087.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:29:29.296 4076 3051113162360 MainThread ERROR nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Failed to change vm state of instance-00000087 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. 2021-09-10 17:29:29.323 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Got request to destroy instance 2021-09-10 17:29:29.323 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-d629c669-b6e2-4fbf-b67e-5791231b21f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:29:29.323 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-d629c669-b6e2-4fbf-b67e-5791231b21f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:29:29.373 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-d629c669-b6e2-4fbf-b67e-5791231b21f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:29:29.386 4076 3051113164120 MainThread INFO nova.compute.manager [req-d629c669-b6e2-4fbf-b67e-5791231b21f8 - - - - -] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:29:31.128 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:29:31.128 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:31.128 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:31.141 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Successfully changed vm state of instance-00000087 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:29:31.141 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Successfully changed state of VM instance-00000087 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:29:31.203 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '24D5C313-00DC-447C-90DD-DA340A20F8D3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14918), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:29:31.341 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job succeeded: {'InstanceID': '24D5C313-00DC-447C-90DD-DA340A20F8D3', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29446), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:29:31.342 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:31.342 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Removing directory: C:\openstack\instances\instance-00000087 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:29:31.342 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:31.342 4076 3051113162360 MainThread ERROR nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Traceback (most recent call last): 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(*args, **kwargs) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] admin_password, network_info, block_device_info) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(self, *args, **kwds) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] should_plug_vifs=False) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'req_state': req_state}) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] time.sleep(sleep_time) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return f(*args, **kwargs) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return self._wait_for_job(job_path) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] error_desc=err_desc) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During handling of the above exception, another exception occurred: 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Traceback (most recent call last): 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] yield resources 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] accel_info=accel_info) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(*args, **kwargs) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] admin_password, network_info, block_device_info) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(self, *args, **kwds) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] should_plug_vifs=False) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'req_state': req_state}) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] time.sleep(sleep_time) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return f(*args, **kwargs) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return self._wait_for_job(job_path) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] error_desc=err_desc) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.342 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.342 4076 3051113162360 MainThread INFO nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Terminating instance 2021-09-10 17:29:31.342 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:29:31.342 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Got request to destroy instance 2021-09-10 17:29:31.361 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:29:31.361 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:31.361 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000087" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:31.361 4076 3051113162360 MainThread INFO nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:29:31.361 4076 3051113162360 MainThread DEBUG nova.compute.claims [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:29:31.361 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:31.459 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:29:31.474 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:29:31.678 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:29:31.686 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:29:31.686 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:31.686 4076 3051113162360 MainThread ERROR nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Traceback (most recent call last): 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(*args, **kwargs) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] admin_password, network_info, block_device_info) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(self, *args, **kwds) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] should_plug_vifs=False) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'req_state': req_state}) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] time.sleep(sleep_time) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return f(*args, **kwargs) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return self._wait_for_job(job_path) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] error_desc=err_desc) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] During handling of the above exception, another exception occurred: 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Traceback (most recent call last): 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] accel_info=accel_info) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(*args, **kwargs) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] admin_password, network_info, block_device_info) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return function(self, *args, **kwds) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] should_plug_vifs=False) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'req_state': req_state}) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] time.sleep(sleep_time) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self.force_reraise() 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] raise self.value 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return f(*args, **kwargs) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] return self._wait_for_job(job_path) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] error_desc=err_desc) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. 2021-09-10 17:29:31.686 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] 2021-09-10 17:29:31.686 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:29:31.704 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Build of instance ff096eb1-eabf-47db-b43c-94d9879eba49 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000087' failed to start. Not enough memory in the system to start the virtual machine instance-00000087. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000087' failed to start. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Not enough memory in the system to start the virtual machine instance-00000087 with ram size 1024 megabytes. (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) 'instance-00000087' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5DB5361A-2F61-4DDB-9E6B-9856B09AD78F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:29:31.704 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:29:31.704 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:29:31.704 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:29:31.704 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:29:32.159 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:32.185 4076 3051113162360 MainThread INFO nova.compute.manager [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ff096eb1-eabf-47db-b43c-94d9879eba49] Took 0.48 seconds to deallocate network for instance. 2021-09-10 17:29:32.344 4076 3051113162360 MainThread INFO nova.scheduler.client.report [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Deleted allocations for instance ff096eb1-eabf-47db-b43c-94d9879eba49 2021-09-10 17:29:32.387 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-579ef65c-265c-49b3-a106-8e680dfcf134 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ff096eb1-eabf-47db-b43c-94d9879eba49" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:53.544 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:53.563 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:29:53.656 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:53.656 4076 3051113163592 MainThread DEBUG nova.virt.hardware [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:29:53.656 4076 3051113163592 MainThread INFO nova.compute.claims [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Claim successful on node n-h1-808474-1 2021-09-10 17:29:53.734 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.hostops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:29:53.773 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.hostops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:29:53.980 4076 3051113163592 MainThread DEBUG nova.compute.provider_tree [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:29:53.989 4076 3051113163592 MainThread DEBUG nova.scheduler.client.report [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:29:53.989 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:53.989 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:29:54.053 4076 3051113163592 MainThread DEBUG nova.compute.utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:29:54.053 4076 3051113164120 GreenThread-124 DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:29:54.053 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:29:54.063 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:29:54.169 4076 3051113164120 GreenThread-124 DEBUG nova.policy [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '652641e6ad894a5cbe0cbe09bd677e6f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ece1a15a5d64ee19aacaadbaf9c9c95', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:29:54.188 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:29:54.188 4076 3051113163592 MainThread INFO nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Spawning new instance 2021-09-10 17:29:54.203 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:54.203 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:54.203 4076 3051113163592 MainThread DEBUG nova.block_device [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:29:54.203 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:54.203 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:54.219 4076 3051113163592 MainThread DEBUG os_win.utils.pathutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Creating directory: C:\openstack\instances\instance-00000088 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:29:54.219 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000088\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:29:54.248 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Resizing VHD C:\openstack\instances\instance-00000088\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:29:54.737 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Successfully created port: 0288a242-301d-4cde-ad88-bcdcb0dedd59 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:29:55.419 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Successfully updated port: 0288a242-301d-4cde-ad88-bcdcb0dedd59 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:29:55.438 4076 3051113164120 GreenThread-124 DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Acquired lock "refresh_cache-ee88e2a3-1118-4d2d-8706-7596a4f77aa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:29:55.438 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:29:55.467 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Received event network-changed-0288a242-301d-4cde-ad88-bcdcb0dedd59 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:29:55.467 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Refreshing instance network info cache due to event network-changed-0288a242-301d-4cde-ad88-bcdcb0dedd59. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:29:55.545 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:29:55.875 4076 3051113164120 GreenThread-124 DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Updating instance_info_cache with network_info: [{"id": "0288a242-301d-4cde-ad88-bcdcb0dedd59", "address": "fa:16:3e:80:6a:ca", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0288a242-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:55.875 4076 3051113164120 GreenThread-124 DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Releasing lock "refresh_cache-ee88e2a3-1118-4d2d-8706-7596a4f77aa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:29:55.875 4076 3051113164120 GreenThread-124 DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Instance network_info: |[{"id": "0288a242-301d-4cde-ad88-bcdcb0dedd59", "address": "fa:16:3e:80:6a:ca", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0288a242-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:29:55.875 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Preparing to wait for external event network-vif-plugged-0288a242-301d-4cde-ad88-bcdcb0dedd59 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:29:55.875 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:29:55.875 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:29:55.875 4076 3051113163592 MainThread DEBUG os_win.utils.compute.vmutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Creating VM instance-00000088 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:29:55.875 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ee88e2a3-1118-4d2d-8706-7596a4f77aa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:29:55.875 4076 3051113160952 MainThread DEBUG nova.network.neutron [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Refreshing network info cache for port 0288a242-301d-4cde-ad88-bcdcb0dedd59 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:29:56.376 4076 3051113160952 MainThread DEBUG nova.network.neutron [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Updated VIF entry in instance network info cache for port 0288a242-301d-4cde-ad88-bcdcb0dedd59. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:29:56.376 4076 3051113160952 MainThread DEBUG nova.network.neutron [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Updating instance_info_cache with network_info: [{"id": "0288a242-301d-4cde-ad88-bcdcb0dedd59", "address": "fa:16:3e:80:6a:ca", "network": {"id": "5bc38b74-b79c-4494-8f9f-215ac9066c53", "bridge": null, "label": "tempest-ServerDiskConfigTestJSON-45318911-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "5ece1a15a5d64ee19aacaadbaf9c9c95", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0288a242-30", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:56.391 4076 3051113160952 MainThread DEBUG oslo_concurrency.lockutils [req-8ea286fc-b313-4f6b-aaf0-f9a98d957ca2 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ee88e2a3-1118-4d2d-8706-7596a4f77aa7" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:29:56.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:29:56.767 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:29:56.779 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:29:56.841 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:29:56.841 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:29:57.562 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:29:57.578 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:29:57.578 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:29:58.778 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:00.531 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-cf60d05f-92ed-4aca-8452-317db492107a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Received event network-vif-plugged-0288a242-301d-4cde-ad88-bcdcb0dedd59 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:30:00.531 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-cf60d05f-92ed-4aca-8452-317db492107a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:00.531 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-cf60d05f-92ed-4aca-8452-317db492107a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:00.531 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-cf60d05f-92ed-4aca-8452-317db492107a 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Processing event network-vif-plugged-0288a242-301d-4cde-ad88-bcdcb0dedd59 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:30:00.566 4076 3051113163592 MainThread INFO nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Using config drive for instance 2021-09-10 17:30:00.656 4076 3051113163592 MainThread INFO nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Creating config drive at C:\openstack\instances\instance-00000088\configdrive.iso 2021-09-10 17:30:00.704 4076 3051113163592 MainThread DEBUG oslo_concurrency.processutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000088\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa3tzhgob execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:30:00.826 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:00.826 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:30:01.013 4076 3051113163592 MainThread DEBUG oslo_concurrency.processutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000088\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpa3tzhgob" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:30:01.392 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Setting boot order for instance: instance-00000088: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:30:01.441 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:30:01.453 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '6F442506-C75A-4E6F-B67E-F612CDCB1CD2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14252), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:01.603 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6F442506-C75A-4E6F-B67E-F612CDCB1CD2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114630), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:01.603 4076 3051113163592 MainThread DEBUG os_win._utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1613321304321289 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:01.603 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:01.622 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:01.622 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:01.622 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:01.637 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d2b06f01-1980-4399-87c2-851ebdb68aff - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:01.637 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-d2b06f01-1980-4399-87c2-851ebdb68aff - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:01.684 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d2b06f01-1980-4399-87c2-851ebdb68aff - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:01.700 4076 3051113162008 MainThread INFO nova.compute.manager [req-d2b06f01-1980-4399-87c2-851ebdb68aff - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:01.762 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:02.622 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': 'EB13952A-BA99-4380-A473-52701388AD18', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15305), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:02.762 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:02.762 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:02.762 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:02.762 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:02.775 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EB13952A-BA99-4380-A473-52701388AD18', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109572), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:02.775 4076 3051113163592 MainThread DEBUG os_win._utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3338935375213623 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:02.787 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-ee9b1b73-89de-4c1d-a694-9812c7201883 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:02.787 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-ee9b1b73-89de-4c1d-a694-9812c7201883 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:02.826 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-ee9b1b73-89de-4c1d-a694-9812c7201883 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:02.859 4076 3051113162008 MainThread INFO nova.compute.manager [req-ee9b1b73-89de-4c1d-a694-9812c7201883 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:03.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:03.811 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '5C851706-0920-4A6E-B3C2-03B040ADF13D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15340), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:03.938 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:03.938 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:03.938 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:03.938 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:03.960 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5C851706-0920-4A6E-B3C2-03B040ADF13D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110649), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:03.960 4076 3051113163592 MainThread DEBUG os_win._utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5190107822418213 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:03.960 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-e9ef2dc4-2a03-4695-b5f3-76257b567d11 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:03.960 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-e9ef2dc4-2a03-4695-b5f3-76257b567d11 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:04.018 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-e9ef2dc4-2a03-4695-b5f3-76257b567d11 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:04.039 4076 3051113162008 MainThread INFO nova.compute.manager [req-e9ef2dc4-2a03-4695-b5f3-76257b567d11 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:04.966 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:04.986 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': '1B01E860-602C-4579-9967-EC2500679C04', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15365), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:05.126 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:05.126 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:05.126 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:05.126 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:05.141 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1B01E860-602C-4579-9967-EC2500679C04', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110892), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:05.141 4076 3051113163592 MainThread DEBUG os_win._utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.699164390563965 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:05.141 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-4dcf1c0e-9b8b-41af-8a89-cdc5c59ed5f3 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:05.141 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-4dcf1c0e-9b8b-41af-8a89-cdc5c59ed5f3 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:05.170 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-4dcf1c0e-9b8b-41af-8a89-cdc5c59ed5f3 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:05.203 4076 3051113162008 MainThread INFO nova.compute.manager [req-4dcf1c0e-9b8b-41af-8a89-cdc5c59ed5f3 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:05.791 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:06.160 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': 'B79C075E-DD1D-4759-807D-5FAF9F50D948', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15603), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:06.278 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:06.278 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:06.278 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:06.302 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:06.312 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B79C075E-DD1D-4759-807D-5FAF9F50D948', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109984), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:06.312 4076 3051113163592 MainThread DEBUG os_win._utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.870218992233276 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:06.312 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-97a8a4c4-508c-49c6-9d51-1a7fd68ce490 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:06.312 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-97a8a4c4-508c-49c6-9d51-1a7fd68ce490 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:06.357 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-97a8a4c4-508c-49c6-9d51-1a7fd68ce490 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:06.378 4076 3051113162008 MainThread INFO nova.compute.manager [req-97a8a4c4-508c-49c6-9d51-1a7fd68ce490 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:06.731 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:06.765 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:06.765 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:06.765 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:06.765 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:30:06.765 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:30:06.794 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:30:07.015 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2199MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:30:07.015 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ee88e2a3-1118-4d2d-8706-7596a4f77aa7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:30:07.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:30:07.189 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:30:07.201 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:30:07.341 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': 'C915CE82-717A-4D7F-9224-236E45FCA9A3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15547), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:07.404 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:30:07.423 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:30:07.423 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:30:07.423 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:07.485 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:07.485 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] VM Stopped (Lifecycle Event) 2021-09-10 17:30:07.485 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:07.485 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:07.504 4076 3051113163592 MainThread ERROR os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000088\' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000088\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)instance-000000882251D82F-C6B7-4AEF-9181-C51520B89AF2%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C915CE82-717A-4D7F-9224-236E45FCA9A3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111934), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\nNot enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)\n\n'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2)", 'ErrorSummaryDescription': "'instance-00000088' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000088.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:07.504 4076 3051113163592 MainThread ERROR nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Failed to change vm state of instance-00000088 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. 2021-09-10 17:30:07.504 4076 3051113163592 MainThread INFO nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Got request to destroy instance 2021-09-10 17:30:07.504 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-0de62404-05f7-48a3-8adb-009fa74953b9 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:07.504 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-0de62404-05f7-48a3-8adb-009fa74953b9 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:07.559 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-0de62404-05f7-48a3-8adb-009fa74953b9 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:07.559 4076 3051113162008 MainThread INFO nova.compute.manager [req-0de62404-05f7-48a3-8adb-009fa74953b9 - - - - -] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:09.296 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:30:09.296 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:09.296 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:09.296 4076 3051113163592 MainThread DEBUG os_win.utils.compute.vmutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Successfully changed vm state of instance-00000088 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:30:09.296 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Successfully changed state of VM instance-00000088 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:30:09.366 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Waiting for WMI job: {'InstanceID': 'C5896B50-8AEB-4278-9E0B-0CCAF1CD19C0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13698), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:09.491 4076 3051113163592 MainThread DEBUG os_win.utils.jobutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] WMI job succeeded: {'InstanceID': 'C5896B50-8AEB-4278-9E0B-0CCAF1CD19C0', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=27764), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:30:09.491 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:09.491 4076 3051113163592 MainThread DEBUG os_win.utils.pathutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Removing directory: C:\openstack\instances\instance-00000088 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:30:09.491 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:09.491 4076 3051113163592 MainThread ERROR nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Traceback (most recent call last): 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(*args, **kwargs) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] admin_password, network_info, block_device_info) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(self, *args, **kwds) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] should_plug_vifs=False) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'req_state': req_state}) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] time.sleep(sleep_time) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return f(*args, **kwargs) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return self._wait_for_job(job_path) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] error_desc=err_desc) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During handling of the above exception, another exception occurred: 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Traceback (most recent call last): 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] yield resources 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] accel_info=accel_info) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(*args, **kwargs) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] admin_password, network_info, block_device_info) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(self, *args, **kwds) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] should_plug_vifs=False) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'req_state': req_state}) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] time.sleep(sleep_time) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return f(*args, **kwargs) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return self._wait_for_job(job_path) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] error_desc=err_desc) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.491 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.491 4076 3051113163592 MainThread INFO nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Terminating instance 2021-09-10 17:30:09.509 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:30:09.509 4076 3051113163592 MainThread INFO nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Got request to destroy instance 2021-09-10 17:30:09.515 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:30:09.515 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:09.515 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "instance-00000088" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:09.515 4076 3051113163592 MainThread INFO nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:30:09.515 4076 3051113163592 MainThread DEBUG nova.compute.claims [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:30:09.531 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:09.613 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.hostops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:30:09.613 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.hostops [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:30:09.825 4076 3051113163592 MainThread DEBUG nova.compute.provider_tree [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:30:09.825 4076 3051113163592 MainThread DEBUG nova.scheduler.client.report [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:30:09.825 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:09.825 4076 3051113163592 MainThread ERROR nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Traceback (most recent call last): 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(*args, **kwargs) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] admin_password, network_info, block_device_info) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(self, *args, **kwds) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] should_plug_vifs=False) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'req_state': req_state}) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] time.sleep(sleep_time) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return f(*args, **kwargs) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return self._wait_for_job(job_path) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] error_desc=err_desc) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] During handling of the above exception, another exception occurred: 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Traceback (most recent call last): 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] accel_info=accel_info) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(*args, **kwargs) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] admin_password, network_info, block_device_info) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return function(self, *args, **kwds) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] should_plug_vifs=False) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'req_state': req_state}) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] time.sleep(sleep_time) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self.force_reraise() 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] raise self.value 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return f(*args, **kwargs) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] return self._wait_for_job(job_path) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] error_desc=err_desc) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. 2021-09-10 17:30:09.825 4076 3051113163592 MainThread TRACE nova.compute.manager [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] 2021-09-10 17:30:09.825 4076 3051113163592 MainThread DEBUG nova.compute.utils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:30:09.825 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Build of instance ee88e2a3-1118-4d2d-8706-7596a4f77aa7 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000088' failed to start. Not enough memory in the system to start the virtual machine instance-00000088. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000088' failed to start. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Not enough memory in the system to start the virtual machine instance-00000088 with ram size 1024 megabytes. (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) 'instance-00000088' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 2251D82F-C6B7-4AEF-9181-C51520B89AF2) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:30:09.856 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:30:09.856 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:30:09.856 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:30:09.856 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:30:10.325 4076 3051113163592 MainThread DEBUG nova.network.neutron [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:30:10.325 4076 3051113163592 MainThread INFO nova.compute.manager [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] [instance: ee88e2a3-1118-4d2d-8706-7596a4f77aa7] Took 0.47 seconds to deallocate network for instance. 2021-09-10 17:30:10.499 4076 3051113163592 MainThread INFO nova.scheduler.client.report [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Deleted allocations for instance ee88e2a3-1118-4d2d-8706-7596a4f77aa7 2021-09-10 17:30:10.532 4076 3051113163592 MainThread DEBUG oslo_concurrency.lockutils [req-02cc0c9d-0f98-410b-b70b-68f9c5b0eaaf 652641e6ad894a5cbe0cbe09bd677e6f 5ece1a15a5d64ee19aacaadbaf9c9c95 - default default] Lock "ee88e2a3-1118-4d2d-8706-7596a4f77aa7" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 16.985s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:11.397 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:39.184 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:39.200 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:30:39.315 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:39.315 4076 3051113164120 MainThread DEBUG nova.virt.hardware [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:30:39.315 4076 3051113164120 MainThread INFO nova.compute.claims [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Claim successful on node n-h1-808474-1 2021-09-10 17:30:39.449 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:30:39.465 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:30:39.673 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:30:39.688 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:30:39.688 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:39.688 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:30:39.746 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:30:39.746 4076 3051113162008 GreenThread-125 DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:30:39.746 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:30:39.777 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:30:39.887 4076 3051113162008 GreenThread-125 DEBUG nova.policy [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '56b46e8aaf0e4c24ad2cbac5d029d70e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c66a56bcea974021b9292220f9d4cade', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:30:39.902 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:30:39.918 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Spawning new instance 2021-09-10 17:30:39.937 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:39.937 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:39.937 4076 3051113164120 MainThread DEBUG nova.block_device [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:30:39.937 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:39.937 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:39.954 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Creating directory: C:\openstack\instances\instance-00000089 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:30:39.954 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000089\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:30:39.980 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Resizing VHD C:\openstack\instances\instance-00000089\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:30:40.548 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Successfully created port: 4667b95a-244e-449f-97ef-9f88b800f559 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:30:41.199 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Successfully updated port: 4667b95a-244e-449f-97ef-9f88b800f559 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:30:41.230 4076 3051113162008 GreenThread-125 DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Acquired lock "refresh_cache-27dff516-729a-44df-9cf5-583985f4ec93" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:30:41.230 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:30:41.268 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Received event network-changed-4667b95a-244e-449f-97ef-9f88b800f559 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:30:41.268 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Refreshing instance network info cache due to event network-changed-4667b95a-244e-449f-97ef-9f88b800f559. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:30:41.308 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:30:41.628 4076 3051113162008 GreenThread-125 DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Updating instance_info_cache with network_info: [{"id": "4667b95a-244e-449f-97ef-9f88b800f559", "address": "fa:16:3e:9a:ca:8c", "network": {"id": "2c384598-3634-4c1c-880c-a730d12034e6", "bridge": null, "label": "tempest-ServerActionsTestJSON-434213352-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "c66a56bcea974021b9292220f9d4cade", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4667b95a-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:30:41.655 4076 3051113162008 GreenThread-125 DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Releasing lock "refresh_cache-27dff516-729a-44df-9cf5-583985f4ec93" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:30:41.655 4076 3051113162008 GreenThread-125 DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Instance network_info: |[{"id": "4667b95a-244e-449f-97ef-9f88b800f559", "address": "fa:16:3e:9a:ca:8c", "network": {"id": "2c384598-3634-4c1c-880c-a730d12034e6", "bridge": null, "label": "tempest-ServerActionsTestJSON-434213352-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "c66a56bcea974021b9292220f9d4cade", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4667b95a-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:30:41.655 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Preparing to wait for external event network-vif-plugged-4667b95a-244e-449f-97ef-9f88b800f559 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:30:41.655 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:41.655 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:41.655 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Creating VM instance-00000089 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:30:41.655 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-27dff516-729a-44df-9cf5-583985f4ec93" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:30:41.655 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Refreshing network info cache for port 4667b95a-244e-449f-97ef-9f88b800f559 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:30:42.137 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Updated VIF entry in instance network info cache for port 4667b95a-244e-449f-97ef-9f88b800f559. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:30:42.137 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Updating instance_info_cache with network_info: [{"id": "4667b95a-244e-449f-97ef-9f88b800f559", "address": "fa:16:3e:9a:ca:8c", "network": {"id": "2c384598-3634-4c1c-880c-a730d12034e6", "bridge": null, "label": "tempest-ServerActionsTestJSON-434213352-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "c66a56bcea974021b9292220f9d4cade", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap4667b95a-24", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:30:42.153 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-b22c2741-85e1-4232-bf4d-c66b21860866 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-27dff516-729a-44df-9cf5-583985f4ec93" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:30:42.537 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:30:46.855 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-f3c16bf3-61d2-4fa5-baf2-f6e9b6185062 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Received event network-vif-plugged-4667b95a-244e-449f-97ef-9f88b800f559 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:30:46.855 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-f3c16bf3-61d2-4fa5-baf2-f6e9b6185062 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:46.855 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-f3c16bf3-61d2-4fa5-baf2-f6e9b6185062 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:46.855 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-f3c16bf3-61d2-4fa5-baf2-f6e9b6185062 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Processing event network-vif-plugged-4667b95a-244e-449f-97ef-9f88b800f559 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:30:46.875 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Using config drive for instance 2021-09-10 17:30:46.985 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Creating config drive at C:\openstack\instances\instance-00000089\configdrive.iso 2021-09-10 17:30:47.031 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000089\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyevqhxq_ execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:30:47.357 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000089\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyevqhxq_" returned: 0 in 0.328s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:30:47.748 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Setting boot order for instance: instance-00000089: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:30:47.782 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:30:47.831 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': 'D25A2786-7FB3-4D76-A4A6-FD864DFA9A43', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18552), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:47.972 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:47.972 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:47.972 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:47.972 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:47.972 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D25A2786-7FB3-4D76-A4A6-FD864DFA9A43', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114902), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:47.972 4076 3051113164120 MainThread DEBUG os_win._utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17052340507507324 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:47.985 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-cd624ad8-8530-4f89-9364-29fd131fb671 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:47.996 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-cd624ad8-8530-4f89-9364-29fd131fb671 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:48.032 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-cd624ad8-8530-4f89-9364-29fd131fb671 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:48.061 4076 3051113160952 MainThread INFO nova.compute.manager [req-cd624ad8-8530-4f89-9364-29fd131fb671 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:48.989 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': '4EB71FF9-DFB9-453D-BC7F-D6CC75F812CC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15289), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:49.126 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:49.126 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:49.126 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:49.126 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:49.140 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4EB71FF9-DFB9-453D-BC7F-D6CC75F812CC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110642), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 48), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 49), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:49.140 4076 3051113164120 MainThread DEBUG os_win._utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3386011123657227 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:49.140 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-79fb6ae4-c04d-44d2-b42d-d2fe28e64607 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:49.140 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-79fb6ae4-c04d-44d2-b42d-d2fe28e64607 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:49.196 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-79fb6ae4-c04d-44d2-b42d-d2fe28e64607 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:49.217 4076 3051113160952 MainThread INFO nova.compute.manager [req-79fb6ae4-c04d-44d2-b42d-d2fe28e64607 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:50.168 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': '2B44891A-2BCE-40C7-81D3-C7438BF84C4D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14513), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:50.297 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:50.297 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:50.297 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:50.297 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:50.309 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2B44891A-2BCE-40C7-81D3-C7438BF84C4D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112703), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 50), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 50), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:50.309 4076 3051113164120 MainThread DEBUG os_win._utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.507943868637085 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:50.309 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-5991d0fa-c272-4a16-866e-cf81d744b465 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:50.309 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-5991d0fa-c272-4a16-866e-cf81d744b465 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:50.356 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-5991d0fa-c272-4a16-866e-cf81d744b465 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:50.374 4076 3051113160952 MainThread INFO nova.compute.manager [req-5991d0fa-c272-4a16-866e-cf81d744b465 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:51.329 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': 'E8664646-3F09-460B-8ED1-82DBFA14097F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15060), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:51.472 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:51.472 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:51.472 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:51.472 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:51.472 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E8664646-3F09-460B-8ED1-82DBFA14097F', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111537), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 51), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 51), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:51.472 4076 3051113164120 MainThread DEBUG os_win._utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6703009605407715 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:51.472 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-b5c93c84-ef15-4eb1-a53a-60ef18055b3b - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:51.472 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-b5c93c84-ef15-4eb1-a53a-60ef18055b3b - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:51.531 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-b5c93c84-ef15-4eb1-a53a-60ef18055b3b - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:51.560 4076 3051113160952 MainThread INFO nova.compute.manager [req-b5c93c84-ef15-4eb1-a53a-60ef18055b3b - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:52.500 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': '9FDA5971-1539-40EB-89AE-BACFD76B6254', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14760), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:52.624 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:52.624 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:52.624 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:52.624 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:52.644 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9FDA5971-1539-40EB-89AE-BACFD76B6254', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111080), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 52), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 52), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:52.644 4076 3051113164120 MainThread DEBUG os_win._utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.842867612838745 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:30:52.658 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-988786a7-ad43-4cb2-b409-e2ea32923e44 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:52.658 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-988786a7-ad43-4cb2-b409-e2ea32923e44 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:52.701 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-988786a7-ad43-4cb2-b409-e2ea32923e44 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:52.726 4076 3051113160952 MainThread INFO nova.compute.manager [req-988786a7-ad43-4cb2-b409-e2ea32923e44 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:53.668 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': '1153807E-E293-41BE-88D8-C8452A7A82EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14827), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:53.799 4076 3051113160952 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:30:53.799 4076 3051113160952 MainThread INFO nova.compute.manager [-] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] VM Stopped (Lifecycle Event) 2021-09-10 17:30:53.799 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:53.799 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:53.819 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job failed: {'RawErrors': (('124\'instance-00000089\' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000089\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)instance-00000089DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1153807E-E293-41BE-88D8-C8452A7A82EA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112545), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 30, 53), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 30, 53), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\nNot enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)\n\n'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C)", 'ErrorSummaryDescription': "'instance-00000089' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000089.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:30:53.819 4076 3051113164120 MainThread ERROR nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Failed to change vm state of instance-00000089 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. 2021-09-10 17:30:53.819 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Got request to destroy instance 2021-09-10 17:30:53.819 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-032bbfd9-e3e6-4d16-8c18-6b38e96162e4 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:30:53.819 4076 3051113160952 MainThread DEBUG nova.virt.hyperv.vmops [req-032bbfd9-e3e6-4d16-8c18-6b38e96162e4 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:30:53.859 4076 3051113160952 MainThread DEBUG nova.compute.manager [req-032bbfd9-e3e6-4d16-8c18-6b38e96162e4 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:30:53.896 4076 3051113160952 MainThread INFO nova.compute.manager [req-032bbfd9-e3e6-4d16-8c18-6b38e96162e4 - - - - -] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:30:55.613 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:30:55.613 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:55.613 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:55.625 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Successfully changed vm state of instance-00000089 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:30:55.625 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Successfully changed state of VM instance-00000089 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:30:55.673 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Waiting for WMI job: {'InstanceID': 'BC57AA8A-11AD-4834-9CAA-BA3CAD4080EF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14613), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:30:55.793 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] WMI job succeeded: {'InstanceID': 'BC57AA8A-11AD-4834-9CAA-BA3CAD4080EF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29815), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:30:55.793 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:55.793 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Removing directory: C:\openstack\instances\instance-00000089 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:30:55.793 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:55.793 4076 3051113164120 MainThread ERROR nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Traceback (most recent call last): 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(*args, **kwargs) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] admin_password, network_info, block_device_info) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(self, *args, **kwds) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] should_plug_vifs=False) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'req_state': req_state}) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] time.sleep(sleep_time) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return f(*args, **kwargs) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return self._wait_for_job(job_path) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] error_desc=err_desc) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During handling of the above exception, another exception occurred: 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Traceback (most recent call last): 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] yield resources 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] accel_info=accel_info) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(*args, **kwargs) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] admin_password, network_info, block_device_info) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(self, *args, **kwds) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] should_plug_vifs=False) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'req_state': req_state}) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] time.sleep(sleep_time) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return f(*args, **kwargs) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return self._wait_for_job(job_path) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] error_desc=err_desc) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:55.793 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:55.793 4076 3051113164120 MainThread INFO nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Terminating instance 2021-09-10 17:30:55.793 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:30:55.793 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Got request to destroy instance 2021-09-10 17:30:55.815 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:30:55.815 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:55.815 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "instance-00000089" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:55.815 4076 3051113164120 MainThread INFO nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Took 0.02 seconds to destroy the instance on the hypervisor. 2021-09-10 17:30:55.815 4076 3051113164120 MainThread DEBUG nova.compute.claims [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:30:55.815 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:30:55.908 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:30:55.922 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:30:56.121 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:30:56.132 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:30:56.132 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:56.132 4076 3051113164120 MainThread ERROR nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Traceback (most recent call last): 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(*args, **kwargs) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] admin_password, network_info, block_device_info) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(self, *args, **kwds) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] should_plug_vifs=False) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'req_state': req_state}) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] time.sleep(sleep_time) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return f(*args, **kwargs) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return self._wait_for_job(job_path) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] error_desc=err_desc) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] During handling of the above exception, another exception occurred: 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Traceback (most recent call last): 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] accel_info=accel_info) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(*args, **kwargs) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] admin_password, network_info, block_device_info) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return function(self, *args, **kwds) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] should_plug_vifs=False) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'req_state': req_state}) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] time.sleep(sleep_time) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self.force_reraise() 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] raise self.value 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return f(*args, **kwargs) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] return self._wait_for_job(job_path) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] error_desc=err_desc) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. 2021-09-10 17:30:56.132 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 27dff516-729a-44df-9cf5-583985f4ec93] 2021-09-10 17:30:56.152 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:30:56.152 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Build of instance 27dff516-729a-44df-9cf5-583985f4ec93 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000089' failed to start. Not enough memory in the system to start the virtual machine instance-00000089. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000089' failed to start. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Not enough memory in the system to start the virtual machine instance-00000089 with ram size 1024 megabytes. (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) 'instance-00000089' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID DECD3AC2-01A4-46CB-BD1C-7EB88CB0D57C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:30:56.152 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:30:56.157 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:30:56.157 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:30:56.157 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:30:56.688 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:30:56.715 4076 3051113164120 MainThread INFO nova.compute.manager [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] [instance: 27dff516-729a-44df-9cf5-583985f4ec93] Took 0.56 seconds to deallocate network for instance. 2021-09-10 17:30:56.890 4076 3051113164120 MainThread INFO nova.scheduler.client.report [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Deleted allocations for instance 27dff516-729a-44df-9cf5-583985f4ec93 2021-09-10 17:30:56.904 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c20f4919-18e0-4c4c-8ca1-04e7a2bb4307 56b46e8aaf0e4c24ad2cbac5d029d70e c66a56bcea974021b9292220f9d4cade - default default] Lock "27dff516-729a-44df-9cf5-583985f4ec93" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 17.719s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:30:57.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:30:57.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:30:57.829 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:30:57.829 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:30:58.356 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:30:58.373 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:30:58.373 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:31:00.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:01.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:01.761 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:31:02.777 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:05.777 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:05.777 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:08.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:08.760 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:08.760 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:08.777 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:08.777 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:31:08.777 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:31:08.781 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:31:09.015 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2213MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:31:09.015 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:31:09.140 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:31:09.172 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:31:09.185 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:31:09.372 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:31:09.409 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:31:09.409 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:31:09.409 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:12.391 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:12.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:24.769 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:24.797 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:31:24.859 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:24.886 4076 3051113162184 MainThread DEBUG nova.virt.hardware [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:31:24.886 4076 3051113162184 MainThread INFO nova.compute.claims [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Claim successful on node n-h1-808474-1 2021-09-10 17:31:24.985 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:31:25.000 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:31:25.187 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:31:25.215 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:31:25.215 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:25.215 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:31:25.264 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:31:25.276 4076 3051113160952 GreenThread-126 DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:31:25.276 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:31:25.276 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:31:25.386 4076 3051113160952 GreenThread-126 DEBUG nova.policy [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '8e710e3097754eeb8ad12e38ed281d7d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8c09d982121a423cbfdf4d85175c464c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:31:25.406 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:31:25.406 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Spawning new instance 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG nova.block_device [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Creating directory: C:\openstack\instances\instance-0000008a check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:31:25.422 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000008a\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:31:25.450 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Resizing VHD C:\openstack\instances\instance-0000008a\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:31:25.953 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Successfully created port: bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:31:26.527 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Successfully updated port: bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:31:26.546 4076 3051113160952 GreenThread-126 DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Acquired lock "refresh_cache-ff2a83fa-db74-482d-8757-342c81429d60" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:31:26.546 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:31:26.563 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Received event network-changed-bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:31:26.563 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Refreshing instance network info cache due to event network-changed-bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:31:26.601 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:31:26.969 4076 3051113160952 GreenThread-126 DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Updating instance_info_cache with network_info: [{"id": "bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea", "address": "fa:16:3e:90:db:63", "network": {"id": "1758ef0b-50dc-45c4-93d4-bc752e87b401", "bridge": null, "label": "tempest-ServersNegativeTestJSON-302151487-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "8c09d982121a423cbfdf4d85175c464c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbcb2b039-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:31:26.984 4076 3051113160952 GreenThread-126 DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Releasing lock "refresh_cache-ff2a83fa-db74-482d-8757-342c81429d60" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:31:26.984 4076 3051113160952 GreenThread-126 DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Instance network_info: |[{"id": "bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea", "address": "fa:16:3e:90:db:63", "network": {"id": "1758ef0b-50dc-45c4-93d4-bc752e87b401", "bridge": null, "label": "tempest-ServersNegativeTestJSON-302151487-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "8c09d982121a423cbfdf4d85175c464c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbcb2b039-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:31:26.984 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Preparing to wait for external event network-vif-plugged-bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:31:26.984 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:26.984 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:26.984 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Creating VM instance-0000008a create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:31:26.984 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-ff2a83fa-db74-482d-8757-342c81429d60" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:31:26.984 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Refreshing network info cache for port bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:31:27.449 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Updated VIF entry in instance network info cache for port bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:31:27.449 4076 3051113161304 MainThread DEBUG nova.network.neutron [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Updating instance_info_cache with network_info: [{"id": "bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea", "address": "fa:16:3e:90:db:63", "network": {"id": "1758ef0b-50dc-45c4-93d4-bc752e87b401", "bridge": null, "label": "tempest-ServersNegativeTestJSON-302151487-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "8c09d982121a423cbfdf4d85175c464c", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapbcb2b039-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:31:27.479 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-a4797129-c5a3-42b2-bbb4-3a5cc7033d5c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-ff2a83fa-db74-482d-8757-342c81429d60" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:31:27.890 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:31:30.998 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-90e55616-6022-4a73-bb91-325a7a635fea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Received event network-vif-plugged-bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:31:30.998 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-90e55616-6022-4a73-bb91-325a7a635fea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:30.998 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [req-90e55616-6022-4a73-bb91-325a7a635fea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:30.998 4076 3051113161304 MainThread DEBUG nova.compute.manager [req-90e55616-6022-4a73-bb91-325a7a635fea 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Processing event network-vif-plugged-bcb2b039-0b88-4b5f-9f02-1e8e2ef571ea _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:31:31.015 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Using config drive for instance 2021-09-10 17:31:31.339 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Creating config drive at C:\openstack\instances\instance-0000008a\configdrive.iso 2021-09-10 17:31:31.403 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphk8kf5fb execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:31:31.700 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008a\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmphk8kf5fb" returned: 0 in 0.297s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:31:32.097 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Setting boot order for instance: instance-0000008a: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:31:32.143 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:31:32.169 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': 'CD0E89A2-38A4-4257-BE89-5422D569C6E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14730), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:32.294 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:32.294 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:32.294 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:32.294 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:32.317 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CD0E89A2-38A4-4257-BE89-5422D569C6E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112890), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:32.317 4076 3051113162184 MainThread DEBUG os_win._utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17419934272766113 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:31:32.328 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-53e4254e-2aba-4f70-888a-a59ba5804cc8 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:32.328 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-53e4254e-2aba-4f70-888a-a59ba5804cc8 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:32.370 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-53e4254e-2aba-4f70-888a-a59ba5804cc8 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:32.401 4076 3051113163592 MainThread INFO nova.compute.manager [req-53e4254e-2aba-4f70-888a-a59ba5804cc8 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:33.359 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '2CA55A0A-BECB-47DD-88B7-7F7ACFCAA8C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17846), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:33.495 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:33.495 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:33.495 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:33.495 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:33.495 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2CA55A0A-BECB-47DD-88B7-7F7ACFCAA8C8', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112445), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:33.495 4076 3051113162184 MainThread DEBUG os_win._utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.352318525314331 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:31:33.495 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-d2d6da9d-322c-40d7-8967-d8f1d46eaad1 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:33.495 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-d2d6da9d-322c-40d7-8967-d8f1d46eaad1 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:33.558 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-d2d6da9d-322c-40d7-8967-d8f1d46eaad1 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:33.558 4076 3051113163592 MainThread INFO nova.compute.manager [req-d2d6da9d-322c-40d7-8967-d8f1d46eaad1 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:34.526 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '96F41E59-5EA4-4543-994A-2D3F13EF2C1D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14985), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:34.668 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:34.668 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:34.668 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:34.668 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:34.668 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '96F41E59-5EA4-4543-994A-2D3F13EF2C1D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109566), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 34), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:34.668 4076 3051113162184 MainThread DEBUG os_win._utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.525362730026245 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:31:34.668 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-b56441a2-6758-49cb-b497-92710e7b2afe - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:34.668 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-b56441a2-6758-49cb-b497-92710e7b2afe - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:34.730 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-b56441a2-6758-49cb-b497-92710e7b2afe - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:34.762 4076 3051113163592 MainThread INFO nova.compute.manager [req-b56441a2-6758-49cb-b497-92710e7b2afe - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:35.685 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '48453311-EFAB-47D1-9820-DA083DCB817D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15383), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:35.817 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:35.817 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:35.834 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:35.834 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:35.834 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '48453311-EFAB-47D1-9820-DA083DCB817D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111181), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 35), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:35.834 4076 3051113162184 MainThread DEBUG os_win._utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6907951831817627 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:31:35.834 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-c3cfb639-02ae-4d0a-97ed-ab924b2c6bb3 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:35.834 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-c3cfb639-02ae-4d0a-97ed-ab924b2c6bb3 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:35.891 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-c3cfb639-02ae-4d0a-97ed-ab924b2c6bb3 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:35.911 4076 3051113163592 MainThread INFO nova.compute.manager [req-c3cfb639-02ae-4d0a-97ed-ab924b2c6bb3 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:36.875 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '5757B7D8-71E6-4E8B-A320-A63CAF6404B2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15388), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:36.997 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:36.997 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:36.997 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:37.020 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.032s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:37.020 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5757B7D8-71E6-4E8B-A320-A63CAF6404B2', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109925), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:37.020 4076 3051113162184 MainThread DEBUG os_win._utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8772664070129395 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:31:37.035 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-885631d7-3201-4d13-a888-b464c8bbada5 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:37.035 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-885631d7-3201-4d13-a888-b464c8bbada5 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:37.075 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-885631d7-3201-4d13-a888-b464c8bbada5 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:37.103 4076 3051113163592 MainThread INFO nova.compute.manager [req-885631d7-3201-4d13-a888-b464c8bbada5 - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:38.052 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '3F2FCED4-F112-4C5E-87DD-A285B350528A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14881), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:38.187 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:38.187 4076 3051113161304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:38.187 4076 3051113163592 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:31:38.187 4076 3051113163592 MainThread INFO nova.compute.manager [-] [instance: ff2a83fa-db74-482d-8757-342c81429d60] VM Stopped (Lifecycle Event) 2021-09-10 17:31:38.198 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008a\' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008a\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)instance-0000008aFF81428F-E83F-4CAF-9764-EA6CAD7C035C%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3F2FCED4-F112-4C5E-87DD-A285B350528A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110774), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 31, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 31, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\nNot enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)\n\n'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C)", 'ErrorSummaryDescription': "'instance-0000008a' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008a.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:31:38.198 4076 3051113162184 MainThread ERROR nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Failed to change vm state of instance-0000008a to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. 2021-09-10 17:31:38.198 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Got request to destroy instance 2021-09-10 17:31:38.215 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-ca7c641d-bd94-41fe-a896-1092d1f3ba4a - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:31:38.215 4076 3051113163592 MainThread DEBUG nova.virt.hyperv.vmops [req-ca7c641d-bd94-41fe-a896-1092d1f3ba4a - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:31:38.251 4076 3051113163592 MainThread DEBUG nova.compute.manager [req-ca7c641d-bd94-41fe-a896-1092d1f3ba4a - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:31:38.281 4076 3051113163592 MainThread INFO nova.compute.manager [req-ca7c641d-bd94-41fe-a896-1092d1f3ba4a - - - - -] [instance: ff2a83fa-db74-482d-8757-342c81429d60] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:31:40.010 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:31:40.015 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:40.015 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:40.015 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Successfully changed vm state of instance-0000008a to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:31:40.015 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Successfully changed state of VM instance-0000008a to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:31:40.074 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Waiting for WMI job: {'InstanceID': '785729D2-1821-4A93-A48D-34A67F2B1345', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13806), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:31:40.183 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] WMI job succeeded: {'InstanceID': '785729D2-1821-4A93-A48D-34A67F2B1345', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29154), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:31:40.183 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:40.183 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Removing directory: C:\openstack\instances\instance-0000008a check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:31:40.183 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:40.183 4076 3051113162184 MainThread ERROR nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Traceback (most recent call last): 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(*args, **kwargs) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] admin_password, network_info, block_device_info) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(self, *args, **kwds) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] should_plug_vifs=False) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'req_state': req_state}) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] time.sleep(sleep_time) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return f(*args, **kwargs) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return self._wait_for_job(job_path) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] error_desc=err_desc) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] During handling of the above exception, another exception occurred: 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Traceback (most recent call last): 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] yield resources 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] accel_info=accel_info) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(*args, **kwargs) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] admin_password, network_info, block_device_info) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(self, *args, **kwds) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] should_plug_vifs=False) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'req_state': req_state}) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] time.sleep(sleep_time) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return f(*args, **kwargs) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return self._wait_for_job(job_path) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] error_desc=err_desc) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.183 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.183 4076 3051113162184 MainThread INFO nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Terminating instance 2021-09-10 17:31:40.183 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:31:40.183 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Got request to destroy instance 2021-09-10 17:31:40.214 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:31:40.214 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:40.214 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "instance-0000008a" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:40.214 4076 3051113162184 MainThread INFO nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:31:40.214 4076 3051113162184 MainThread DEBUG nova.compute.claims [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:31:40.229 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:31:40.317 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:31:40.334 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:31:40.533 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:40.548 4076 3051113162184 MainThread ERROR nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Traceback (most recent call last): 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(*args, **kwargs) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] admin_password, network_info, block_device_info) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(self, *args, **kwds) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] should_plug_vifs=False) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'req_state': req_state}) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] time.sleep(sleep_time) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return f(*args, **kwargs) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return self._wait_for_job(job_path) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] error_desc=err_desc) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] During handling of the above exception, another exception occurred: 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Traceback (most recent call last): 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] accel_info=accel_info) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(*args, **kwargs) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] admin_password, network_info, block_device_info) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return function(self, *args, **kwds) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] should_plug_vifs=False) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'req_state': req_state}) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] time.sleep(sleep_time) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self.force_reraise() 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] raise self.value 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return f(*args, **kwargs) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] return self._wait_for_job(job_path) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] error_desc=err_desc) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. 2021-09-10 17:31:40.548 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: ff2a83fa-db74-482d-8757-342c81429d60] 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Build of instance ff2a83fa-db74-482d-8757-342c81429d60 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008a' failed to start. Not enough memory in the system to start the virtual machine instance-0000008a. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008a' failed to start. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Not enough memory in the system to start the virtual machine instance-0000008a with ram size 1024 megabytes. (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) 'instance-0000008a' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID FF81428F-E83F-4CAF-9764-EA6CAD7C035C) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:31:40.548 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:31:41.089 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:31:41.115 4076 3051113162184 MainThread INFO nova.compute.manager [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] [instance: ff2a83fa-db74-482d-8757-342c81429d60] Took 0.58 seconds to deallocate network for instance. 2021-09-10 17:31:41.276 4076 3051113162184 MainThread INFO nova.scheduler.client.report [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Deleted allocations for instance ff2a83fa-db74-482d-8757-342c81429d60 2021-09-10 17:31:41.295 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-2f91dbf8-648a-49aa-81f6-4d080e85f69d 8e710e3097754eeb8ad12e38ed281d7d 8c09d982121a423cbfdf4d85175c464c - default default] Lock "ff2a83fa-db74-482d-8757-342c81429d60" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 16.531s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:31:58.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:31:58.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:31:58.845 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:31:58.845 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:31:59.395 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:31:59.413 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:31:59.413 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:32:01.760 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:01.760 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:01.760 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:32:03.760 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:05.775 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:05.967 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:06.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:06.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:09.725 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:09.762 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:09.762 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:09.762 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:09.762 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:32:09.762 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:32:09.802 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:32:10.018 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:32:10.018 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:32:10.321 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:32:10.400 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:32:10.432 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:32:10.432 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:32:10.465 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:32:10.511 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:32:10.511 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:32:10.511 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:32:10.713 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:32:10.729 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:32:10.729 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:32:10.729 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.719s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:15.733 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:19.760 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:19.760 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:32:30.958 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:30.963 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:32:31.063 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:31.063 4076 3051113162184 MainThread DEBUG nova.virt.hardware [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:32:31.063 4076 3051113162184 MainThread INFO nova.compute.claims [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Claim successful on node n-h1-808474-1 2021-09-10 17:32:31.173 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:32:31.189 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:32:31.371 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:32:31.403 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:32:31.403 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:31.403 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:32:31.464 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:32:31.464 4076 3051113162360 GreenThread-127 DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:32:31.464 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:32:31.464 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:32:31.582 4076 3051113162360 GreenThread-127 DEBUG nova.policy [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0226fc9bb37e4645999ab1e146569f72', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1a659c58dcc3425b8ccbfd6f1f95de43', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:32:31.619 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:32:31.619 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Spawning new instance 2021-09-10 17:32:31.619 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:31.619 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:31.650 4076 3051113162184 MainThread DEBUG nova.block_device [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:32:31.650 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:31.650 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:31.656 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Creating directory: C:\openstack\instances\instance-0000008b check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:32:31.656 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000008b\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:32:31.682 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Resizing VHD C:\openstack\instances\instance-0000008b\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:32:32.507 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Successfully created port: 9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:32:33.258 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Successfully updated port: 9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:32:33.265 4076 3051113162360 GreenThread-127 DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Acquired lock "refresh_cache-27fe0257-cfa7-41ce-8543-5349d843a088" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:32:33.279 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:32:33.298 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Received event network-changed-9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:32:33.298 4076 3051113294312 MainThread DEBUG nova.compute.manager [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Refreshing instance network info cache due to event network-changed-9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:32:33.359 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:32:33.682 4076 3051113162360 GreenThread-127 DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Updating instance_info_cache with network_info: [{"id": "9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a", "address": "fa:16:3e:e1:85:82", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9fdcfdb8-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:32:33.712 4076 3051113162360 GreenThread-127 DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Releasing lock "refresh_cache-27fe0257-cfa7-41ce-8543-5349d843a088" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:32:33.712 4076 3051113162360 GreenThread-127 DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Instance network_info: |[{"id": "9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a", "address": "fa:16:3e:e1:85:82", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9fdcfdb8-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:32:33.712 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Preparing to wait for external event network-vif-plugged-9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:32:33.712 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:33.712 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:33.712 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Creating VM instance-0000008b create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:32:33.712 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-27fe0257-cfa7-41ce-8543-5349d843a088" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:32:33.712 4076 3051113294312 MainThread DEBUG nova.network.neutron [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Refreshing network info cache for port 9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:32:34.260 4076 3051113294312 MainThread DEBUG nova.network.neutron [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Updated VIF entry in instance network info cache for port 9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:32:34.260 4076 3051113294312 MainThread DEBUG nova.network.neutron [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Updating instance_info_cache with network_info: [{"id": "9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a", "address": "fa:16:3e:e1:85:82", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9fdcfdb8-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:32:34.260 4076 3051113294312 MainThread DEBUG oslo_concurrency.lockutils [req-80e834b9-4a77-42ec-9251-14cd34b0b92f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-27fe0257-cfa7-41ce-8543-5349d843a088" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:32:34.603 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:32:37.314 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-891bd556-9e53-402e-a20b-9fd5efcd1135 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Received event network-vif-plugged-9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:32:37.323 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-891bd556-9e53-402e-a20b-9fd5efcd1135 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:37.323 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-891bd556-9e53-402e-a20b-9fd5efcd1135 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:37.323 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-891bd556-9e53-402e-a20b-9fd5efcd1135 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Processing event network-vif-plugged-9fdcfdb8-e9d6-4e93-933e-c8a3e7e69f5a _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:32:37.345 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Using config drive for instance 2021-09-10 17:32:37.462 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Creating config drive at C:\openstack\instances\instance-0000008b\configdrive.iso 2021-09-10 17:32:37.485 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpwza01sww execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:32:37.796 4076 3051113162184 MainThread DEBUG oslo_concurrency.processutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008b\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpwza01sww" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:32:38.182 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Setting boot order for instance: instance-0000008b: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:32:38.239 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:32:38.266 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '4F14258E-F30C-4B09-B187-50CE9809FB19', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19000), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:38.403 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:38.403 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:38.403 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:38.403 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:38.422 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4F14258E-F30C-4B09-B187-50CE9809FB19', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113200), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:38.422 4076 3051113162184 MainThread DEBUG os_win._utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18359160423278809 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:32:38.422 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-e9e960f6-2928-4a39-87f4-3b77b0266921 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:38.422 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-e9e960f6-2928-4a39-87f4-3b77b0266921 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:38.469 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-e9e960f6-2928-4a39-87f4-3b77b0266921 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:38.489 4076 3051113162360 MainThread INFO nova.compute.manager [req-e9e960f6-2928-4a39-87f4-3b77b0266921 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:39.447 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': 'FD1EE2FA-C74E-45B5-8575-626BE11F25DA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14723), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:39.579 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:39.579 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:39.579 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:39.579 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:39.595 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FD1EE2FA-C74E-45B5-8575-626BE11F25DA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110971), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:39.595 4076 3051113162184 MainThread DEBUG os_win._utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.355797529220581 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:32:39.595 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c67580a4-0b6c-4e9e-8d58-6a651fd0411e - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:39.595 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-c67580a4-0b6c-4e9e-8d58-6a651fd0411e - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:39.648 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c67580a4-0b6c-4e9e-8d58-6a651fd0411e - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:39.658 4076 3051113162360 MainThread INFO nova.compute.manager [req-c67580a4-0b6c-4e9e-8d58-6a651fd0411e - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:39.790 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:39.790 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:32:39.794 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:32:40.620 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '0E0CF565-A3DF-4C83-982C-FA5CBC02878D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14869), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:40.745 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:40.745 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:40.745 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:40.745 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:40.762 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0E0CF565-A3DF-4C83-982C-FA5CBC02878D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=108683), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:40.762 4076 3051113162184 MainThread DEBUG os_win._utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5230777263641357 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:32:40.762 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-46060c62-866e-46aa-a7dc-4918fe2eeb41 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:40.762 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-46060c62-866e-46aa-a7dc-4918fe2eeb41 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:40.814 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-46060c62-866e-46aa-a7dc-4918fe2eeb41 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:40.837 4076 3051113162360 MainThread INFO nova.compute.manager [req-46060c62-866e-46aa-a7dc-4918fe2eeb41 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:41.806 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '19434F0B-B3EB-480F-B424-CB90CEECF231', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15442), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:41.922 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:41.922 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:41.922 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:41.922 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:41.947 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-a1e67fd0-d8d4-4849-bd06-f0822f47c649 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:41.947 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-a1e67fd0-d8d4-4849-bd06-f0822f47c649 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:41.947 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '19434F0B-B3EB-480F-B424-CB90CEECF231', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110037), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:41.947 4076 3051113162184 MainThread DEBUG os_win._utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7087721824645996 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:32:42.006 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-a1e67fd0-d8d4-4849-bd06-f0822f47c649 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:42.025 4076 3051113162360 MainThread INFO nova.compute.manager [req-a1e67fd0-d8d4-4849-bd06-f0822f47c649 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:42.963 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': 'DBB87946-D0D5-44AF-AB09-1005C6BFF344', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14569), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:43.103 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:43.103 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:43.103 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:43.103 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:43.103 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DBB87946-D0D5-44AF-AB09-1005C6BFF344', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110168), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:43.103 4076 3051113162184 MainThread DEBUG os_win._utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.864178657531738 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:32:43.126 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-4373c652-4df6-49e1-b957-13b8fa39c08c - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:43.126 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-4373c652-4df6-49e1-b957-13b8fa39c08c - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:43.159 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-4373c652-4df6-49e1-b957-13b8fa39c08c - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:43.196 4076 3051113162360 MainThread INFO nova.compute.manager [req-4373c652-4df6-49e1-b957-13b8fa39c08c - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:44.136 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '4ED5A153-FC44-4CAA-8369-E616FEEBF381', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15623), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:44.265 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:44.265 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:44.275 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:32:44.275 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] VM Stopped (Lifecycle Event) 2021-09-10 17:32:44.275 4076 3051113162184 MainThread ERROR os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008b\' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81212030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD81210243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008b\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)instance-0000008bD6F6D71C-3B24-4812-846D-292DA98CD812%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4ED5A153-FC44-4CAA-8369-E616FEEBF381', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109638), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 32, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 32, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\nNot enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)\n\n'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812)", 'ErrorSummaryDescription': "'instance-0000008b' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008b.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:32:44.275 4076 3051113162184 MainThread ERROR nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Failed to change vm state of instance-0000008b to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. 2021-09-10 17:32:44.275 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Got request to destroy instance 2021-09-10 17:32:44.275 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-fafe37ff-3fb6-4ee4-8e65-f3454d379754 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:32:44.295 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-fafe37ff-3fb6-4ee4-8e65-f3454d379754 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:32:44.342 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-fafe37ff-3fb6-4ee4-8e65-f3454d379754 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:32:44.363 4076 3051113162360 MainThread INFO nova.compute.manager [req-fafe37ff-3fb6-4ee4-8e65-f3454d379754 - - - - -] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:32:46.072 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:32:46.072 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:46.072 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:46.072 4076 3051113162184 MainThread DEBUG os_win.utils.compute.vmutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Successfully changed vm state of instance-0000008b to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:32:46.072 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Successfully changed state of VM instance-0000008b to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:32:46.141 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': 'AAF4F82C-EC13-47B6-806F-19E90079E4BF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14673), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:32:46.259 4076 3051113162184 MainThread DEBUG os_win.utils.jobutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job succeeded: {'InstanceID': 'AAF4F82C-EC13-47B6-806F-19E90079E4BF', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29991), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:32:46.259 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:46.259 4076 3051113162184 MainThread DEBUG os_win.utils.pathutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Removing directory: C:\openstack\instances\instance-0000008b check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:32:46.259 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:46.259 4076 3051113162184 MainThread ERROR nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Traceback (most recent call last): 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(*args, **kwargs) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] admin_password, network_info, block_device_info) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(self, *args, **kwds) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] should_plug_vifs=False) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'req_state': req_state}) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] time.sleep(sleep_time) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return f(*args, **kwargs) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return self._wait_for_job(job_path) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] error_desc=err_desc) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During handling of the above exception, another exception occurred: 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Traceback (most recent call last): 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] yield resources 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] accel_info=accel_info) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(*args, **kwargs) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] admin_password, network_info, block_device_info) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(self, *args, **kwds) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] should_plug_vifs=False) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'req_state': req_state}) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] time.sleep(sleep_time) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return f(*args, **kwargs) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return self._wait_for_job(job_path) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] error_desc=err_desc) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.259 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.259 4076 3051113162184 MainThread INFO nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Terminating instance 2021-09-10 17:32:46.259 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:32:46.259 4076 3051113162184 MainThread INFO nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Got request to destroy instance 2021-09-10 17:32:46.289 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:32:46.289 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:46.289 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008b" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:46.289 4076 3051113162184 MainThread INFO nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:32:46.306 4076 3051113162184 MainThread DEBUG nova.compute.claims [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:32:46.306 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:32:46.375 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:32:46.407 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.hostops [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:32:46.602 4076 3051113162184 MainThread DEBUG nova.compute.provider_tree [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:32:46.616 4076 3051113162184 MainThread DEBUG nova.scheduler.client.report [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:32:46.631 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:46.631 4076 3051113162184 MainThread ERROR nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Traceback (most recent call last): 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(*args, **kwargs) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] admin_password, network_info, block_device_info) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(self, *args, **kwds) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] should_plug_vifs=False) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'req_state': req_state}) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] time.sleep(sleep_time) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return f(*args, **kwargs) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return self._wait_for_job(job_path) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] error_desc=err_desc) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] During handling of the above exception, another exception occurred: 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Traceback (most recent call last): 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] accel_info=accel_info) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(*args, **kwargs) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] admin_password, network_info, block_device_info) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return function(self, *args, **kwds) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] should_plug_vifs=False) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'req_state': req_state}) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] time.sleep(sleep_time) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self.force_reraise() 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] raise self.value 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return f(*args, **kwargs) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] return self._wait_for_job(job_path) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] error_desc=err_desc) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. 2021-09-10 17:32:46.631 4076 3051113162184 MainThread TRACE nova.compute.manager [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] 2021-09-10 17:32:46.631 4076 3051113162184 MainThread DEBUG nova.compute.utils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:32:46.631 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Build of instance 27fe0257-cfa7-41ce-8543-5349d843a088 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008b' failed to start. Not enough memory in the system to start the virtual machine instance-0000008b. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008b' failed to start. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Not enough memory in the system to start the virtual machine instance-0000008b with ram size 1024 megabytes. (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) 'instance-0000008b' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D6F6D71C-3B24-4812-846D-292DA98CD812) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:32:46.640 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:32:46.640 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:32:46.640 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:32:46.640 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:32:47.181 4076 3051113162184 MainThread DEBUG nova.network.neutron [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:32:47.218 4076 3051113162184 MainThread INFO nova.compute.manager [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: 27fe0257-cfa7-41ce-8543-5349d843a088] Took 0.58 seconds to deallocate network for instance. 2021-09-10 17:32:47.384 4076 3051113162184 MainThread INFO nova.scheduler.client.report [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Deleted allocations for instance 27fe0257-cfa7-41ce-8543-5349d843a088 2021-09-10 17:32:47.415 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-c99a6ab2-d349-4163-b60b-937e525a801a 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "27fe0257-cfa7-41ce-8543-5349d843a088" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 16.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:32:58.766 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:32:58.766 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:32:58.837 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:32:58.837 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:32:59.296 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:32:59.325 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:32:59.325 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:33:02.755 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:02.755 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:02.755 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:33:05.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:07.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:07.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:11.743 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:11.743 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:11.789 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:11.796 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:11.796 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:11.796 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:33:11.796 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:33:11.821 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:33:12.028 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2213MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:33:12.028 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:33:12.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:33:12.212 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:33:12.212 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:33:12.417 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:33:12.431 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:33:12.431 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:33:12.431 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:13.045 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:13.087 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:33:13.197 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:13.197 4076 3051113162360 MainThread DEBUG nova.virt.hardware [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:33:13.197 4076 3051113162360 MainThread INFO nova.compute.claims [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Claim successful on node n-h1-808474-1 2021-09-10 17:33:13.305 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:33:13.328 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:33:13.521 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:33:13.531 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:33:13.531 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:13.531 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:33:13.605 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:33:13.605 4076 3051113162184 GreenThread-128 DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:33:13.605 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:33:13.625 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:33:13.715 4076 3051113162184 GreenThread-128 DEBUG nova.policy [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '0226fc9bb37e4645999ab1e146569f72', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1a659c58dcc3425b8ccbfd6f1f95de43', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:33:13.715 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:33:13.743 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Spawning new instance 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG nova.block_device [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Creating directory: C:\openstack\instances\instance-0000008c check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:33:13.760 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000008c\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:33:13.790 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Resizing VHD C:\openstack\instances\instance-0000008c\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:33:14.290 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Successfully created port: 262820ad-2a80-4d25-85f3-32d186ac77f9 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:33:14.899 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Successfully updated port: 262820ad-2a80-4d25-85f3-32d186ac77f9 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:33:14.922 4076 3051113162184 GreenThread-128 DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Acquired lock "refresh_cache-d2676f87-5c64-40cc-b7e7-a75b36bfbe57" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:33:14.922 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:33:14.954 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Received event network-changed-262820ad-2a80-4d25-85f3-32d186ac77f9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:33:14.954 4076 3051113293784 MainThread DEBUG nova.compute.manager [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Refreshing instance network info cache due to event network-changed-262820ad-2a80-4d25-85f3-32d186ac77f9. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:33:15.033 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:33:15.375 4076 3051113162184 GreenThread-128 DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Updating instance_info_cache with network_info: [{"id": "262820ad-2a80-4d25-85f3-32d186ac77f9", "address": "fa:16:3e:49:d9:c6", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap262820ad-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:33:15.441 4076 3051113162184 GreenThread-128 DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Releasing lock "refresh_cache-d2676f87-5c64-40cc-b7e7-a75b36bfbe57" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:33:15.441 4076 3051113162184 GreenThread-128 DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Instance network_info: |[{"id": "262820ad-2a80-4d25-85f3-32d186ac77f9", "address": "fa:16:3e:49:d9:c6", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap262820ad-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:33:15.441 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Preparing to wait for external event network-vif-plugged-262820ad-2a80-4d25-85f3-32d186ac77f9 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:33:15.441 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:15.441 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:15.441 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Creating VM instance-0000008c create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:33:15.441 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d2676f87-5c64-40cc-b7e7-a75b36bfbe57" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:33:15.441 4076 3051113293784 MainThread DEBUG nova.network.neutron [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Refreshing network info cache for port 262820ad-2a80-4d25-85f3-32d186ac77f9 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:33:15.922 4076 3051113293784 MainThread DEBUG nova.network.neutron [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Updated VIF entry in instance network info cache for port 262820ad-2a80-4d25-85f3-32d186ac77f9. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:33:15.922 4076 3051113293784 MainThread DEBUG nova.network.neutron [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Updating instance_info_cache with network_info: [{"id": "262820ad-2a80-4d25-85f3-32d186ac77f9", "address": "fa:16:3e:49:d9:c6", "network": {"id": "32eb283c-70ec-49c4-a64b-fe78dcc445e4", "bridge": null, "label": "tempest-TestEncryptedCinderVolumes-1170647147-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "1a659c58dcc3425b8ccbfd6f1f95de43", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap262820ad-2a", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:33:15.937 4076 3051113293784 MainThread DEBUG oslo_concurrency.lockutils [req-3214e9fb-7e35-4c76-aa78-b6091a4554b5 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d2676f87-5c64-40cc-b7e7-a75b36bfbe57" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:33:16.309 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:33:17.406 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:33:19.619 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-ed375940-584d-4560-8cd7-175f99a0720b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Received event network-vif-plugged-262820ad-2a80-4d25-85f3-32d186ac77f9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:33:19.619 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-ed375940-584d-4560-8cd7-175f99a0720b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:19.619 4076 3051113162184 MainThread DEBUG oslo_concurrency.lockutils [req-ed375940-584d-4560-8cd7-175f99a0720b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:19.619 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-ed375940-584d-4560-8cd7-175f99a0720b 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Processing event network-vif-plugged-262820ad-2a80-4d25-85f3-32d186ac77f9 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:33:19.672 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Using config drive for instance 2021-09-10 17:33:19.839 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Creating config drive at C:\openstack\instances\instance-0000008c\configdrive.iso 2021-09-10 17:33:19.869 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkdolsv8c execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:33:20.180 4076 3051113162360 MainThread DEBUG oslo_concurrency.processutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008c\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpkdolsv8c" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:33:20.557 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Setting boot order for instance: instance-0000008c: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:33:20.605 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:33:20.635 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '43C2001E-12B2-4B4A-8BAA-9E32C8350934', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15299), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:20.779 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:20.779 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:20.779 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:20.779 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:20.779 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '43C2001E-12B2-4B4A-8BAA-9E32C8350934', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117620), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:20.779 4076 3051113162360 MainThread DEBUG os_win._utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.17326617240905762 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:33:20.797 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-a120ebbb-059d-49cd-afdf-f5d14c6370f6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:20.797 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-a120ebbb-059d-49cd-afdf-f5d14c6370f6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:20.840 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-a120ebbb-059d-49cd-afdf-f5d14c6370f6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:20.860 4076 3051113162184 MainThread INFO nova.compute.manager [req-a120ebbb-059d-49cd-afdf-f5d14c6370f6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:21.809 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '2DC68526-FB42-410E-BAC5-4C8A8F67F87B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14428), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:21.944 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:21.944 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:21.944 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:21.944 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:21.963 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '2DC68526-FB42-410E-BAC5-4C8A8F67F87B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112039), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:21.963 4076 3051113162360 MainThread DEBUG os_win._utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3570911884307861 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:33:21.963 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-00ca9a28-b569-4fc2-ae25-831c9cee81c6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:21.963 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-00ca9a28-b569-4fc2-ae25-831c9cee81c6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:22.010 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-00ca9a28-b569-4fc2-ae25-831c9cee81c6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:22.039 4076 3051113162184 MainThread INFO nova.compute.manager [req-00ca9a28-b569-4fc2-ae25-831c9cee81c6 - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:22.989 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': 'AE62AEA9-7468-410E-B3FC-3E1D68F2DCD0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14742), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:23.121 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:23.121 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:23.121 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:23.121 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:23.141 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-38b4a0a7-b45d-4210-a7bf-e56365b5417e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:23.141 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-38b4a0a7-b45d-4210-a7bf-e56365b5417e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:23.141 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'AE62AEA9-7468-410E-B3FC-3E1D68F2DCD0', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112950), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 23), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:23.156 4076 3051113162360 MainThread DEBUG os_win._utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.551055669784546 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:33:23.183 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-38b4a0a7-b45d-4210-a7bf-e56365b5417e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:23.212 4076 3051113162184 MainThread INFO nova.compute.manager [req-38b4a0a7-b45d-4210-a7bf-e56365b5417e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:24.180 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '7865186A-A8D6-44C1-AE0E-C48D44ECBABE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14858), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:24.314 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:24.314 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:24.314 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:24.314 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:24.328 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-06d80520-aa8e-47bd-a32b-2135a365c81d - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:24.328 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-06d80520-aa8e-47bd-a32b-2135a365c81d - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:24.328 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '7865186A-A8D6-44C1-AE0E-C48D44ECBABE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110386), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 24), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:24.328 4076 3051113162360 MainThread DEBUG os_win._utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.722895860671997 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:33:24.368 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-06d80520-aa8e-47bd-a32b-2135a365c81d - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:24.392 4076 3051113162184 MainThread INFO nova.compute.manager [req-06d80520-aa8e-47bd-a32b-2135a365c81d - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:25.337 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '811638C5-FCA6-437C-88C2-0398ADE76B98', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15123), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:25.479 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '811638C5-FCA6-437C-88C2-0398ADE76B98', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110239), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:25.479 4076 3051113162360 MainThread DEBUG os_win._utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.873634338378906 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:33:25.479 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:25.479 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:25.479 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:25.504 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:25.504 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-9c5da660-c19c-4bf8-a260-64b7d1411c8a - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:25.504 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-9c5da660-c19c-4bf8-a260-64b7d1411c8a - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:25.548 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-9c5da660-c19c-4bf8-a260-64b7d1411c8a - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:25.589 4076 3051113162184 MainThread INFO nova.compute.manager [req-9c5da660-c19c-4bf8-a260-64b7d1411c8a - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:26.508 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': 'A09B597D-CF84-4D81-8CB2-436412B4B18A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15297), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:26.657 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:26.657 4076 3051113162008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:26.657 4076 3051113162184 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:33:26.657 4076 3051113162184 MainThread INFO nova.compute.manager [-] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] VM Stopped (Lifecycle Event) 2021-09-10 17:33:26.657 4076 3051113162360 MainThread ERROR os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008c\' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008c\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)instance-0000008c763ACB24-B774-4E62-8191-50C7A670B27B%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A09B597D-CF84-4D81-8CB2-436412B4B18A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109893), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 33, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 33, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\nNot enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)\n\n'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B)", 'ErrorSummaryDescription': "'instance-0000008c' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008c.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:33:26.657 4076 3051113162360 MainThread ERROR nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Failed to change vm state of instance-0000008c to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. 2021-09-10 17:33:26.657 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Got request to destroy instance 2021-09-10 17:33:26.680 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-069fb473-89e7-4cc6-9a63-af6bf3cec94e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:33:26.680 4076 3051113162184 MainThread DEBUG nova.virt.hyperv.vmops [req-069fb473-89e7-4cc6-9a63-af6bf3cec94e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:33:26.727 4076 3051113162184 MainThread DEBUG nova.compute.manager [req-069fb473-89e7-4cc6-9a63-af6bf3cec94e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:33:26.750 4076 3051113162184 MainThread INFO nova.compute.manager [req-069fb473-89e7-4cc6-9a63-af6bf3cec94e - - - - -] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:33:28.477 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:33:28.477 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:28.477 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:28.487 4076 3051113162360 MainThread DEBUG os_win.utils.compute.vmutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Successfully changed vm state of instance-0000008c to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:33:28.487 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Successfully changed state of VM instance-0000008c to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:33:28.526 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Waiting for WMI job: {'InstanceID': '23203CFE-EA03-4CD5-9D4D-2033167D590F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14452), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:33:28.636 4076 3051113162360 MainThread DEBUG os_win.utils.jobutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] WMI job succeeded: {'InstanceID': '23203CFE-EA03-4CD5-9D4D-2033167D590F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29127), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:33:28.636 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:28.636 4076 3051113162360 MainThread DEBUG os_win.utils.pathutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Removing directory: C:\openstack\instances\instance-0000008c check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:33:28.636 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:28.636 4076 3051113162360 MainThread ERROR nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Traceback (most recent call last): 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(*args, **kwargs) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] admin_password, network_info, block_device_info) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(self, *args, **kwds) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] should_plug_vifs=False) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'req_state': req_state}) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] time.sleep(sleep_time) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return f(*args, **kwargs) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return self._wait_for_job(job_path) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] error_desc=err_desc) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During handling of the above exception, another exception occurred: 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Traceback (most recent call last): 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] yield resources 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] accel_info=accel_info) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(*args, **kwargs) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] admin_password, network_info, block_device_info) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(self, *args, **kwds) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] should_plug_vifs=False) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'req_state': req_state}) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] time.sleep(sleep_time) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return f(*args, **kwargs) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return self._wait_for_job(job_path) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] error_desc=err_desc) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.636 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.657 4076 3051113162360 MainThread INFO nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Terminating instance 2021-09-10 17:33:28.657 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:33:28.657 4076 3051113162360 MainThread INFO nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Got request to destroy instance 2021-09-10 17:33:28.673 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:33:28.673 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:28.673 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "instance-0000008c" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:28.673 4076 3051113162360 MainThread INFO nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 17:33:28.673 4076 3051113162360 MainThread DEBUG nova.compute.claims [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:33:28.673 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:33:28.767 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:33:28.776 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.hostops [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:33:28.985 4076 3051113162360 MainThread DEBUG nova.compute.provider_tree [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.scheduler.client.report [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:33:28.993 4076 3051113162360 MainThread ERROR nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Traceback (most recent call last): 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(*args, **kwargs) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] admin_password, network_info, block_device_info) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(self, *args, **kwds) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] should_plug_vifs=False) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'req_state': req_state}) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] time.sleep(sleep_time) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return f(*args, **kwargs) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return self._wait_for_job(job_path) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] error_desc=err_desc) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] During handling of the above exception, another exception occurred: 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Traceback (most recent call last): 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] accel_info=accel_info) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(*args, **kwargs) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] admin_password, network_info, block_device_info) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return function(self, *args, **kwds) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] should_plug_vifs=False) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'req_state': req_state}) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] time.sleep(sleep_time) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self.force_reraise() 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] raise self.value 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return f(*args, **kwargs) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] return self._wait_for_job(job_path) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] error_desc=err_desc) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. 2021-09-10 17:33:28.993 4076 3051113162360 MainThread TRACE nova.compute.manager [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.compute.utils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Build of instance d2676f87-5c64-40cc-b7e7-a75b36bfbe57 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008c' failed to start. Not enough memory in the system to start the virtual machine instance-0000008c. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008c' failed to start. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Not enough memory in the system to start the virtual machine instance-0000008c with ram size 1024 megabytes. (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) 'instance-0000008c' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 763ACB24-B774-4E62-8191-50C7A670B27B) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:33:28.993 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:33:29.517 4076 3051113162360 MainThread DEBUG nova.network.neutron [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:33:29.543 4076 3051113162360 MainThread INFO nova.compute.manager [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] [instance: d2676f87-5c64-40cc-b7e7-a75b36bfbe57] Took 0.55 seconds to deallocate network for instance. 2021-09-10 17:33:29.711 4076 3051113162360 MainThread INFO nova.scheduler.client.report [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Deleted allocations for instance d2676f87-5c64-40cc-b7e7-a75b36bfbe57 2021-09-10 17:33:29.745 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-8b135ca7-015d-41d8-beb2-c1e081e08a0d 0226fc9bb37e4645999ab1e146569f72 1a659c58dcc3425b8ccbfd6f1f95de43 - default default] Lock "d2676f87-5c64-40cc-b7e7-a75b36bfbe57" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 16.703s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:00.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:00.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:34:00.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 17:34:00.932 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:34:00.932 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:34:00.932 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 17:34:01.492 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:34:01.517 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:34:01.517 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:34:02.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:02.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:02.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:34:07.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:08.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:09.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:09.954 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:11.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:13.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:13.760 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:13.760 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:13.760 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:34:13.760 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:34:13.798 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:34:14.007 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2214MB free_disk=51GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:34:14.007 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:14.156 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.156 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.156 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.156 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:34:14.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:34:14.189 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:34:14.211 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:34:14.406 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:34:14.429 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:34:14.429 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:34:14.429 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:15.765 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:15.776 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:34:15.858 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:15.882 4076 3051113293080 MainThread DEBUG nova.virt.hardware [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:34:15.884 4076 3051113293080 MainThread INFO nova.compute.claims [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Claim successful on node n-h1-808474-1 2021-09-10 17:34:15.983 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:34:15.983 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:34:16.194 4076 3051113293080 MainThread DEBUG nova.compute.provider_tree [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:34:16.205 4076 3051113293080 MainThread DEBUG nova.scheduler.client.report [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:34:16.205 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:16.205 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:34:16.267 4076 3051113293080 MainThread DEBUG nova.compute.utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:34:16.267 4076 3051113162360 GreenThread-129 DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:34:16.267 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:34:16.283 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:34:16.377 4076 3051113162360 GreenThread-129 DEBUG nova.policy [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ffb5dbb2ad6e47068976b3e937dc4322', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f601013bf0c474aac57e1ac136b5136', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:34:16.401 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:34:16.429 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Spawning new instance 2021-09-10 17:34:16.429 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:16.429 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:16.429 4076 3051113293080 MainThread DEBUG nova.block_device [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:34:16.429 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "f2cce4b1-ac64-485e-8426-63b59c59ab51-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:28.257 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "f2cce4b1-ac64-485e-8426-63b59c59ab51-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 11.828s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:28.257 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:34:28.257 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Successfully created port: 2f148ca3-bb2d-4de9-8e2f-03f089e7610b _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:34:28.340 4076 3051113293080 MainThread DEBUG os_win.utils.pathutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Creating directory: C:\openstack\instances\instance-0000008d check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:34:28.340 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Creating differencing VHD. Parent: C:\openstack\instances\_base\f2cce4b1-ac64-485e-8426-63b59c59ab51.vhdx, Target: C:\openstack\instances\instance-0000008d\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:34:28.477 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Resizing VHD C:\openstack\instances\instance-0000008d\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:34:29.150 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Successfully updated port: 2f148ca3-bb2d-4de9-8e2f-03f089e7610b _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:34:29.164 4076 3051113162360 GreenThread-129 DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Acquired lock "refresh_cache-6233922c-7a39-4d26-a5b3-44889c3647c3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:34:29.164 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:34:29.194 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Received event network-changed-2f148ca3-bb2d-4de9-8e2f-03f089e7610b external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:34:29.194 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Refreshing instance network info cache due to event network-changed-2f148ca3-bb2d-4de9-8e2f-03f089e7610b. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:34:29.241 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:34:29.564 4076 3051113162360 GreenThread-129 DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Updating instance_info_cache with network_info: [{"id": "2f148ca3-bb2d-4de9-8e2f-03f089e7610b", "address": "fa:16:3e:b6:ea:d6", "network": {"id": "1b7bf981-f3de-4768-a09e-78fe8ea37484", "bridge": null, "label": "tempest-TestMinimumBasicScenario-1484923914-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "9f601013bf0c474aac57e1ac136b5136", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2f148ca3-bb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:34:29.585 4076 3051113162360 GreenThread-129 DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Releasing lock "refresh_cache-6233922c-7a39-4d26-a5b3-44889c3647c3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:34:29.585 4076 3051113162360 GreenThread-129 DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Instance network_info: |[{"id": "2f148ca3-bb2d-4de9-8e2f-03f089e7610b", "address": "fa:16:3e:b6:ea:d6", "network": {"id": "1b7bf981-f3de-4768-a09e-78fe8ea37484", "bridge": null, "label": "tempest-TestMinimumBasicScenario-1484923914-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "9f601013bf0c474aac57e1ac136b5136", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2f148ca3-bb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:34:29.585 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-6233922c-7a39-4d26-a5b3-44889c3647c3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:34:29.585 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Refreshing network info cache for port 2f148ca3-bb2d-4de9-8e2f-03f089e7610b _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:34:29.585 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Preparing to wait for external event network-vif-plugged-2f148ca3-bb2d-4de9-8e2f-03f089e7610b prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:34:29.585 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:29.585 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:29.585 4076 3051113293080 MainThread DEBUG os_win.utils.compute.vmutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Creating VM instance-0000008d create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:34:30.137 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Updated VIF entry in instance network info cache for port 2f148ca3-bb2d-4de9-8e2f-03f089e7610b. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:34:30.137 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Updating instance_info_cache with network_info: [{"id": "2f148ca3-bb2d-4de9-8e2f-03f089e7610b", "address": "fa:16:3e:b6:ea:d6", "network": {"id": "1b7bf981-f3de-4768-a09e-78fe8ea37484", "bridge": null, "label": "tempest-TestMinimumBasicScenario-1484923914-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "9f601013bf0c474aac57e1ac136b5136", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2f148ca3-bb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:34:30.163 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [req-a2e4f77e-ae76-4656-989c-6acc27344cbc 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-6233922c-7a39-4d26-a5b3-44889c3647c3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:34:30.463 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:34:35.961 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-482601d3-5ce6-45da-9af0-8a184b849d6c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Received event network-vif-plugged-2f148ca3-bb2d-4de9-8e2f-03f089e7610b external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:34:35.961 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-482601d3-5ce6-45da-9af0-8a184b849d6c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:35.961 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-482601d3-5ce6-45da-9af0-8a184b849d6c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:35.961 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-482601d3-5ce6-45da-9af0-8a184b849d6c 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Processing event network-vif-plugged-2f148ca3-bb2d-4de9-8e2f-03f089e7610b _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:34:35.994 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Using config drive for instance 2021-09-10 17:34:36.096 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Creating config drive at C:\openstack\instances\instance-0000008d\configdrive.iso 2021-09-10 17:34:36.118 4076 3051113293080 MainThread DEBUG oslo_concurrency.processutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpzgxfv0y3 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:34:36.440 4076 3051113293080 MainThread DEBUG oslo_concurrency.processutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008d\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpzgxfv0y3" returned: 0 in 0.329s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:34:36.820 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Setting boot order for instance: instance-0000008d: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:34:36.873 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:34:36.901 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': 'B0235A3C-18EB-46A7-9895-969B3EB9D2E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18749), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:37.039 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:37.039 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:37.039 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:37.039 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:37.039 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B0235A3C-18EB-46A7-9895-969B3EB9D2E6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113255), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:37.039 4076 3051113293080 MainThread DEBUG os_win._utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16584515571594238 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:34:37.059 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-a80ea4cb-3823-4d03-bd65-0b723e339ca1 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:37.059 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-a80ea4cb-3823-4d03-bd65-0b723e339ca1 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:37.109 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-a80ea4cb-3823-4d03-bd65-0b723e339ca1 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:37.132 4076 3051113162360 MainThread INFO nova.compute.manager [req-a80ea4cb-3823-4d03-bd65-0b723e339ca1 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:38.066 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': 'B8D3E866-392E-4E63-B62C-70273A96A25C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14736), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:38.195 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:38.195 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:38.195 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:38.195 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:38.220 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3da3c8a6-64f1-4257-b52d-4f2e7dab50a9 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:38.220 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-3da3c8a6-64f1-4257-b52d-4f2e7dab50a9 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:38.220 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'B8D3E866-392E-4E63-B62C-70273A96A25C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111487), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 38), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 38), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:38.220 4076 3051113293080 MainThread DEBUG os_win._utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3465325832366943 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:34:38.260 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-3da3c8a6-64f1-4257-b52d-4f2e7dab50a9 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:38.295 4076 3051113162360 MainThread INFO nova.compute.manager [req-3da3c8a6-64f1-4257-b52d-4f2e7dab50a9 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:39.250 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': '57136927-3443-4736-8FC3-D75B4C90B947', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15731), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:39.389 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:39.389 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:39.389 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:39.389 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:39.399 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '57136927-3443-4736-8FC3-D75B4C90B947', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113237), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 39), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 39), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:39.399 4076 3051113293080 MainThread DEBUG os_win._utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.525946855545044 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:34:39.399 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d712efaf-8de2-463b-b423-8e3cca85bbcc - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:39.417 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-d712efaf-8de2-463b-b423-8e3cca85bbcc - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:39.464 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d712efaf-8de2-463b-b423-8e3cca85bbcc - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:39.484 4076 3051113162360 MainThread INFO nova.compute.manager [req-d712efaf-8de2-463b-b423-8e3cca85bbcc - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:40.414 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': '6CBA8AA3-440B-4F6B-BB35-78A577C2E7ED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15346), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:40.557 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:40.557 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:40.557 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:40.557 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:40.557 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '6CBA8AA3-440B-4F6B-BB35-78A577C2E7ED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109468), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:40.557 4076 3051113293080 MainThread DEBUG os_win._utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6840009689331055 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:34:40.557 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c4708714-18f7-4446-82bf-f79c44039276 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:40.581 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-c4708714-18f7-4446-82bf-f79c44039276 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:40.616 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c4708714-18f7-4446-82bf-f79c44039276 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:40.643 4076 3051113162360 MainThread INFO nova.compute.manager [req-c4708714-18f7-4446-82bf-f79c44039276 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:41.574 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': 'BD19947D-297D-42F5-A768-EFCE1E958D0A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15105), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:41.697 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:41.697 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:41.717 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:41.717 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:41.717 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BD19947D-297D-42F5-A768-EFCE1E958D0A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=109534), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:41.717 4076 3051113293080 MainThread DEBUG os_win._utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.8437440395355225 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:34:41.717 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d7a80327-424b-428f-ac59-9f4c9058ca42 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:41.717 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-d7a80327-424b-428f-ac59-9f4c9058ca42 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:41.769 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d7a80327-424b-428f-ac59-9f4c9058ca42 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:41.803 4076 3051113162360 MainThread INFO nova.compute.manager [req-d7a80327-424b-428f-ac59-9f4c9058ca42 - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:42.746 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': 'EFB6B047-F485-44FE-9938-F936EC82BC9D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15174), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:42.883 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:34:42.883 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] VM Stopped (Lifecycle Event) 2021-09-10 17:34:42.883 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:42.883 4076 3051113294840 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:42.883 4076 3051113293080 MainThread ERROR os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008d\' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476912030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F476910243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008d\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)instance-0000008dD26FBC5D-44A5-48F4-AEC5-38C0788F4769%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EFB6B047-F485-44FE-9938-F936EC82BC9D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110772), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 34, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 34, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\nNot enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)\n\n'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769)", 'ErrorSummaryDescription': "'instance-0000008d' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008d.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:34:42.883 4076 3051113293080 MainThread ERROR nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Failed to change vm state of instance-0000008d to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. 2021-09-10 17:34:42.883 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Got request to destroy instance 2021-09-10 17:34:42.908 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b4568775-6058-4d07-aa9c-a887463037ae - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:34:42.908 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-b4568775-6058-4d07-aa9c-a887463037ae - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:34:42.947 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b4568775-6058-4d07-aa9c-a887463037ae - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:34:42.975 4076 3051113162360 MainThread INFO nova.compute.manager [req-b4568775-6058-4d07-aa9c-a887463037ae - - - - -] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:34:44.694 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:34:44.694 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:44.694 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:44.699 4076 3051113293080 MainThread DEBUG os_win.utils.compute.vmutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Successfully changed vm state of instance-0000008d to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:34:44.699 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Successfully changed state of VM instance-0000008d to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:34:44.755 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Waiting for WMI job: {'InstanceID': '2F9E08C5-BA4B-4CFD-AF52-2C549EBB06D2', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13974), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:34:44.879 4076 3051113293080 MainThread DEBUG os_win.utils.jobutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] WMI job succeeded: {'InstanceID': '2F9E08C5-BA4B-4CFD-AF52-2C549EBB06D2', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29929), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:34:44.879 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:44.879 4076 3051113293080 MainThread DEBUG os_win.utils.pathutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Removing directory: C:\openstack\instances\instance-0000008d check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:34:44.879 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:44.879 4076 3051113293080 MainThread ERROR nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Traceback (most recent call last): 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(*args, **kwargs) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] admin_password, network_info, block_device_info) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(self, *args, **kwds) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] should_plug_vifs=False) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'req_state': req_state}) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] time.sleep(sleep_time) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return f(*args, **kwargs) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return self._wait_for_job(job_path) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] error_desc=err_desc) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During handling of the above exception, another exception occurred: 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Traceback (most recent call last): 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] yield resources 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] accel_info=accel_info) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(*args, **kwargs) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] admin_password, network_info, block_device_info) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(self, *args, **kwds) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] should_plug_vifs=False) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'req_state': req_state}) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] time.sleep(sleep_time) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return f(*args, **kwargs) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return self._wait_for_job(job_path) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] error_desc=err_desc) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:44.879 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:44.891 4076 3051113293080 MainThread INFO nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Terminating instance 2021-09-10 17:34:44.891 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:34:44.891 4076 3051113293080 MainThread INFO nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Got request to destroy instance 2021-09-10 17:34:44.898 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.vmops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:34:44.898 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:44.898 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "instance-0000008d" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:44.898 4076 3051113293080 MainThread INFO nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:34:44.898 4076 3051113293080 MainThread DEBUG nova.compute.claims [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:34:44.898 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:34:44.995 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:34:45.007 4076 3051113293080 MainThread DEBUG nova.virt.hyperv.hostops [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:34:45.212 4076 3051113293080 MainThread DEBUG nova.compute.provider_tree [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.scheduler.client.report [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:34:45.226 4076 3051113293080 MainThread ERROR nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Traceback (most recent call last): 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(*args, **kwargs) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] admin_password, network_info, block_device_info) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(self, *args, **kwds) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] should_plug_vifs=False) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'req_state': req_state}) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] time.sleep(sleep_time) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return f(*args, **kwargs) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return self._wait_for_job(job_path) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] error_desc=err_desc) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] During handling of the above exception, another exception occurred: 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Traceback (most recent call last): 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] accel_info=accel_info) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(*args, **kwargs) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] admin_password, network_info, block_device_info) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return function(self, *args, **kwds) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] should_plug_vifs=False) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'req_state': req_state}) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] time.sleep(sleep_time) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self.force_reraise() 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] raise self.value 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return f(*args, **kwargs) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] return self._wait_for_job(job_path) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] error_desc=err_desc) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. 2021-09-10 17:34:45.226 4076 3051113293080 MainThread TRACE nova.compute.manager [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.compute.utils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Build of instance 6233922c-7a39-4d26-a5b3-44889c3647c3 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008d' failed to start. Not enough memory in the system to start the virtual machine instance-0000008d. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008d' failed to start. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Not enough memory in the system to start the virtual machine instance-0000008d with ram size 1024 megabytes. (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) 'instance-0000008d' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID D26FBC5D-44A5-48F4-AEC5-38C0788F4769) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:34:45.226 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:34:45.703 4076 3051113293080 MainThread DEBUG nova.network.neutron [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:34:45.738 4076 3051113293080 MainThread INFO nova.compute.manager [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] [instance: 6233922c-7a39-4d26-a5b3-44889c3647c3] Took 0.51 seconds to deallocate network for instance. 2021-09-10 17:34:45.957 4076 3051113293080 MainThread INFO nova.scheduler.client.report [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Deleted allocations for instance 6233922c-7a39-4d26-a5b3-44889c3647c3 2021-09-10 17:34:45.988 4076 3051113293080 MainThread DEBUG oslo_concurrency.lockutils [req-95ae4e2f-1b10-40f4-953e-02ae240bd838 ffb5dbb2ad6e47068976b3e937dc4322 9f601013bf0c474aac57e1ac136b5136 - default default] Lock "6233922c-7a39-4d26-a5b3-44889c3647c3" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 30.234s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:00.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:00.769 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:35:00.957 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:35:00.957 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:35:01.439 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:35:01.443 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:35:01.443 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:35:02.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:02.754 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:35:03.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:07.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:10.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:11.718 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:11.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:12.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:13.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:13.778 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:13.778 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:13.778 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:35:13.778 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:35:13.798 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:35:14.021 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:35:14.021 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.146 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.162 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:35:14.162 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:35:14.162 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:35:14.186 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:35:14.194 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:35:14.390 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:35:14.425 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:35:14.425 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:35:14.425 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:17.396 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:35:32.863 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:32.878 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:35:32.975 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:32.975 4076 3051113293256 MainThread DEBUG nova.virt.hardware [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:35:32.975 4076 3051113293256 MainThread INFO nova.compute.claims [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Claim successful on node n-h1-808474-1 2021-09-10 17:35:33.084 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:35:33.110 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:35:33.336 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:35:33.365 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:35:33.365 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:33.381 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:35:33.443 4076 3051113293256 MainThread DEBUG nova.compute.utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:35:33.443 4076 3051113162360 GreenThread-130 DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:35:33.443 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:35:33.474 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:35:33.572 4076 3051113162360 GreenThread-130 DEBUG nova.policy [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '580888399edc4d7c97b4a1bce73db000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '351b14c1ba6b4a35aa1c79568c8e5c30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:35:33.628 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:35:33.652 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Spawning new instance 2021-09-10 17:35:33.652 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:33.652 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:33.652 4076 3051113293256 MainThread DEBUG nova.block_device [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:35:33.652 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:33.652 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:33.677 4076 3051113293256 MainThread DEBUG os_win.utils.pathutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating directory: C:\openstack\instances\instance-0000008e check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:35:33.677 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000008e\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:35:33.693 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Resizing VHD C:\openstack\instances\instance-0000008e\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:35:34.265 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Successfully created port: e94da4cb-0b2b-40c8-b8d9-571c1e503a5f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:35:34.958 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Successfully updated port: e94da4cb-0b2b-40c8-b8d9-571c1e503a5f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:35:34.975 4076 3051113162360 GreenThread-130 DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Acquired lock "refresh_cache-d96429b4-66c9-4d52-8c01-f9c377be61ff" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:35:34.975 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:35:35.052 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Received event network-changed-e94da4cb-0b2b-40c8-b8d9-571c1e503a5f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:35:35.052 4076 3051113293960 MainThread DEBUG nova.compute.manager [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Refreshing instance network info cache due to event network-changed-e94da4cb-0b2b-40c8-b8d9-571c1e503a5f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:35:35.099 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:35:35.431 4076 3051113162360 GreenThread-130 DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Updating instance_info_cache with network_info: [{"id": "e94da4cb-0b2b-40c8-b8d9-571c1e503a5f", "address": "fa:16:3e:86:b9:87", "network": {"id": "a74f7a61-699f-45dd-8e25-3e73b17b8d87", "bridge": null, "label": "tempest-network-smoke--847854339", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape94da4cb-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:35:35.445 4076 3051113162360 GreenThread-130 DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Releasing lock "refresh_cache-d96429b4-66c9-4d52-8c01-f9c377be61ff" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:35:35.445 4076 3051113162360 GreenThread-130 DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Instance network_info: |[{"id": "e94da4cb-0b2b-40c8-b8d9-571c1e503a5f", "address": "fa:16:3e:86:b9:87", "network": {"id": "a74f7a61-699f-45dd-8e25-3e73b17b8d87", "bridge": null, "label": "tempest-network-smoke--847854339", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape94da4cb-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:35:35.445 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-d96429b4-66c9-4d52-8c01-f9c377be61ff" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:35:35.445 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Refreshing network info cache for port e94da4cb-0b2b-40c8-b8d9-571c1e503a5f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:35:35.445 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Preparing to wait for external event network-vif-plugged-e94da4cb-0b2b-40c8-b8d9-571c1e503a5f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:35:35.445 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:35.445 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:35.445 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating VM instance-0000008e create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:35:36.198 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Updated VIF entry in instance network info cache for port e94da4cb-0b2b-40c8-b8d9-571c1e503a5f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:35:36.198 4076 3051113293960 MainThread DEBUG nova.network.neutron [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Updating instance_info_cache with network_info: [{"id": "e94da4cb-0b2b-40c8-b8d9-571c1e503a5f", "address": "fa:16:3e:86:b9:87", "network": {"id": "a74f7a61-699f-45dd-8e25-3e73b17b8d87", "bridge": null, "label": "tempest-network-smoke--847854339", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tape94da4cb-0b", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:35:36.198 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [req-966365eb-334f-47cc-a504-91b76aa1a7cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-d96429b4-66c9-4d52-8c01-f9c377be61ff" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:35:36.333 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:35:40.317 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-7406e2a7-6677-42c5-a979-498f742db4d1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Received event network-vif-plugged-e94da4cb-0b2b-40c8-b8d9-571c1e503a5f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:35:40.317 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-7406e2a7-6677-42c5-a979-498f742db4d1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:40.317 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-7406e2a7-6677-42c5-a979-498f742db4d1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:40.317 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-7406e2a7-6677-42c5-a979-498f742db4d1 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Processing event network-vif-plugged-e94da4cb-0b2b-40c8-b8d9-571c1e503a5f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:35:40.352 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Using config drive for instance 2021-09-10 17:35:40.473 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Creating config drive at C:\openstack\instances\instance-0000008e\configdrive.iso 2021-09-10 17:35:40.505 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyimnrfyo execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:35:40.821 4076 3051113293256 MainThread DEBUG oslo_concurrency.processutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008e\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpyimnrfyo" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:35:41.197 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Setting boot order for instance: instance-0000008e: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:35:41.245 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:35:41.271 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '3B0C66E9-3A94-467B-B704-8EB2975F95E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19094), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:41.413 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:41.413 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:41.413 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:41.413 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:41.413 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '3B0C66E9-3A94-467B-B704-8EB2975F95E9', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114587), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 41), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 41), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:41.413 4076 3051113293256 MainThread DEBUG os_win._utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.16790366172790527 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:35:41.413 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-552692f7-633b-41f4-b739-5a6417e85e8b - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:41.413 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-552692f7-633b-41f4-b739-5a6417e85e8b - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:41.489 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-552692f7-633b-41f4-b739-5a6417e85e8b - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:41.515 4076 3051113162360 MainThread INFO nova.compute.manager [req-552692f7-633b-41f4-b739-5a6417e85e8b - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:42.431 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'EFB5F735-3E55-4840-9238-B94250654F03', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15295), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:42.574 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:42.574 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:42.574 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:42.574 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:42.574 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'EFB5F735-3E55-4840-9238-B94250654F03', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112799), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 42), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 42), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:42.574 4076 3051113293256 MainThread DEBUG os_win._utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.328587293624878 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:35:42.574 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b3c17bd6-51c9-4332-a48b-2dcb409c3962 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:42.574 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-b3c17bd6-51c9-4332-a48b-2dcb409c3962 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:42.638 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b3c17bd6-51c9-4332-a48b-2dcb409c3962 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:42.660 4076 3051113162360 MainThread INFO nova.compute.manager [req-b3c17bd6-51c9-4332-a48b-2dcb409c3962 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:43.612 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'E4946CAC-47CE-4A13-90F3-F5E4BDB64DCA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15302), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:43.746 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:43.746 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:43.746 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:43.746 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:43.746 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E4946CAC-47CE-4A13-90F3-F5E4BDB64DCA', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115275), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 43), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 43), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:43.746 4076 3051113293256 MainThread DEBUG os_win._utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5006279945373535 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:35:43.772 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-9d479ece-b7ed-4d7e-9047-396383d29045 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:43.772 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-9d479ece-b7ed-4d7e-9047-396383d29045 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:43.815 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-9d479ece-b7ed-4d7e-9047-396383d29045 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:43.833 4076 3051113162360 MainThread INFO nova.compute.manager [req-9d479ece-b7ed-4d7e-9047-396383d29045 - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:44.758 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'C6EED0DD-4BF6-46B5-890F-762DF55A8422', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15183), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:44.893 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:44.893 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:44.893 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:44.893 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:44.911 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'C6EED0DD-4BF6-46B5-890F-762DF55A8422', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112058), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 44), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 44), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:44.911 4076 3051113293256 MainThread DEBUG os_win._utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6664488315582275 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:35:44.911 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-04b67ab9-1cec-429c-9572-eb2dd6c07f6d - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:44.911 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-04b67ab9-1cec-429c-9572-eb2dd6c07f6d - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:44.970 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-04b67ab9-1cec-429c-9572-eb2dd6c07f6d - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:44.989 4076 3051113162360 MainThread INFO nova.compute.manager [req-04b67ab9-1cec-429c-9572-eb2dd6c07f6d - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:45.937 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '61028D62-50B3-403F-8485-8BA543DD2A66', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17642), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:46.065 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:46.065 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:46.065 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:46.078 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.015s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:46.093 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d113a84d-2469-4d02-a52a-79904c123d9c - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:46.093 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-d113a84d-2469-4d02-a52a-79904c123d9c - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:46.093 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '61028D62-50B3-403F-8485-8BA543DD2A66', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110803), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 45), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 46), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:46.093 4076 3051113293256 MainThread DEBUG os_win._utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.847842216491699 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:35:46.117 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-d113a84d-2469-4d02-a52a-79904c123d9c - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:46.162 4076 3051113162360 MainThread INFO nova.compute.manager [req-d113a84d-2469-4d02-a52a-79904c123d9c - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:47.130 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'FCE0F545-DB66-4C46-875F-8A1BD24148F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14269), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:47.262 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:47.262 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:47.262 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:35:47.262 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] VM Stopped (Lifecycle Event) 2021-09-10 17:35:47.262 4076 3051113293256 MainThread ERROR os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008e\' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008e\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)instance-0000008e54504C15-EC7A-4385-811E-B0DEA9A57E5F%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FCE0F545-DB66-4C46-875F-8A1BD24148F7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111286), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 35, 47), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 35, 47), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\nNot enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)\n\n'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F)", 'ErrorSummaryDescription': "'instance-0000008e' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008e.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:35:47.262 4076 3051113293256 MainThread ERROR nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Failed to change vm state of instance-0000008e to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. 2021-09-10 17:35:47.262 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Got request to destroy instance 2021-09-10 17:35:47.281 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-30bea5fe-d06a-468a-abe6-395bd8737fdd - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:35:47.281 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-30bea5fe-d06a-468a-abe6-395bd8737fdd - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:35:47.318 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-30bea5fe-d06a-468a-abe6-395bd8737fdd - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:35:47.351 4076 3051113162360 MainThread INFO nova.compute.manager [req-30bea5fe-d06a-468a-abe6-395bd8737fdd - - - - -] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:35:49.077 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:35:49.077 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:49.077 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:49.099 4076 3051113293256 MainThread DEBUG os_win.utils.compute.vmutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed vm state of instance-0000008e to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:35:49.099 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed state of VM instance-0000008e to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:35:49.139 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '088CC437-7F04-4220-BEA1-3553E9096B96', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14246), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:35:49.263 4076 3051113293256 MainThread DEBUG os_win.utils.jobutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job succeeded: {'InstanceID': '088CC437-7F04-4220-BEA1-3553E9096B96', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28376), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:35:49.263 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG os_win.utils.pathutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Removing directory: C:\openstack\instances\instance-0000008e check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:49.271 4076 3051113293256 MainThread ERROR nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Traceback (most recent call last): 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(*args, **kwargs) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] admin_password, network_info, block_device_info) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(self, *args, **kwds) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] should_plug_vifs=False) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'req_state': req_state}) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] time.sleep(sleep_time) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return f(*args, **kwargs) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return self._wait_for_job(job_path) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] error_desc=err_desc) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During handling of the above exception, another exception occurred: 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Traceback (most recent call last): 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] yield resources 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] accel_info=accel_info) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(*args, **kwargs) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] admin_password, network_info, block_device_info) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(self, *args, **kwds) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] should_plug_vifs=False) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'req_state': req_state}) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] time.sleep(sleep_time) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return f(*args, **kwargs) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return self._wait_for_job(job_path) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] error_desc=err_desc) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.271 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.271 4076 3051113293256 MainThread INFO nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Terminating instance 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:35:49.271 4076 3051113293256 MainThread INFO nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Got request to destroy instance 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.vmops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:49.271 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008e" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:49.271 4076 3051113293256 MainThread INFO nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:35:49.299 4076 3051113293256 MainThread DEBUG nova.compute.claims [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:35:49.299 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:35:49.380 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:35:49.405 4076 3051113293256 MainThread DEBUG nova.virt.hyperv.hostops [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:35:49.600 4076 3051113293256 MainThread DEBUG nova.compute.provider_tree [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.scheduler.client.report [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:35:49.615 4076 3051113293256 MainThread ERROR nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Traceback (most recent call last): 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(*args, **kwargs) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] admin_password, network_info, block_device_info) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(self, *args, **kwds) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] should_plug_vifs=False) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'req_state': req_state}) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] time.sleep(sleep_time) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return f(*args, **kwargs) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return self._wait_for_job(job_path) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] error_desc=err_desc) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] During handling of the above exception, another exception occurred: 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Traceback (most recent call last): 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] accel_info=accel_info) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(*args, **kwargs) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] admin_password, network_info, block_device_info) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return function(self, *args, **kwds) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] should_plug_vifs=False) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'req_state': req_state}) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] time.sleep(sleep_time) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self.force_reraise() 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] raise self.value 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return f(*args, **kwargs) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] return self._wait_for_job(job_path) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] error_desc=err_desc) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. 2021-09-10 17:35:49.615 4076 3051113293256 MainThread TRACE nova.compute.manager [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.compute.utils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Build of instance d96429b4-66c9-4d52-8c01-f9c377be61ff was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008e' failed to start. Not enough memory in the system to start the virtual machine instance-0000008e. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008e' failed to start. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Not enough memory in the system to start the virtual machine instance-0000008e with ram size 1024 megabytes. (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) 'instance-0000008e' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 54504C15-EC7A-4385-811E-B0DEA9A57E5F) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:35:49.615 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:35:50.160 4076 3051113293256 MainThread DEBUG nova.network.neutron [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:35:50.175 4076 3051113293256 MainThread INFO nova.compute.manager [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: d96429b4-66c9-4d52-8c01-f9c377be61ff] Took 0.56 seconds to deallocate network for instance. 2021-09-10 17:35:50.353 4076 3051113293256 MainThread INFO nova.scheduler.client.report [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Deleted allocations for instance d96429b4-66c9-4d52-8c01-f9c377be61ff 2021-09-10 17:35:50.389 4076 3051113293256 MainThread DEBUG oslo_concurrency.lockutils [req-1c126ff5-518e-4dc6-b2b3-073483c62c5c 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "d96429b4-66c9-4d52-8c01-f9c377be61ff" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 17.532s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:01.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:01.758 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:36:01.817 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:36:01.817 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:36:02.318 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:36:02.333 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:36:02.333 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:36:03.757 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:04.767 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:04.767 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:36:08.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:10.765 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:10.957 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:12.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:13.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:15.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:15.773 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:15.773 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:15.773 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:36:15.773 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:36:15.773 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:36:16.009 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:36:16.009 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:36:16.161 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:36:16.204 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:36:16.216 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:36:16.404 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:36:16.427 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:36:16.427 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:36:16.427 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.422s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:17.422 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:36:23.394 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:23.408 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:36:23.509 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:23.509 4076 3051113295016 MainThread DEBUG nova.virt.hardware [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:36:23.509 4076 3051113295016 MainThread INFO nova.compute.claims [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Claim successful on node n-h1-808474-1 2021-09-10 17:36:23.599 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:36:23.647 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:36:23.849 4076 3051113295016 MainThread DEBUG nova.compute.provider_tree [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:36:23.849 4076 3051113295016 MainThread DEBUG nova.scheduler.client.report [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:36:23.849 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:23.849 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:36:23.925 4076 3051113295016 MainThread DEBUG nova.compute.utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:36:23.925 4076 3051113162360 GreenThread-131 DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:36:23.925 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:36:23.952 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:36:24.083 4076 3051113162360 GreenThread-131 DEBUG nova.policy [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '580888399edc4d7c97b4a1bce73db000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '351b14c1ba6b4a35aa1c79568c8e5c30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:36:24.135 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:36:24.135 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Spawning new instance 2021-09-10 17:36:24.135 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG nova.block_device [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG os_win.utils.pathutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating directory: C:\openstack\instances\instance-0000008f check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:36:24.156 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-0000008f\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:36:24.194 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Resizing VHD C:\openstack\instances\instance-0000008f\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:36:24.795 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Successfully created port: b80b94a3-c7d4-4b87-80b7-3d673800f7e9 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:36:25.584 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Successfully updated port: b80b94a3-c7d4-4b87-80b7-3d673800f7e9 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:36:25.602 4076 3051113162360 GreenThread-131 DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Acquired lock "refresh_cache-749d3e34-a2bc-4962-a645-dbcbfc91b72f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:36:25.602 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:36:25.635 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Received event network-changed-b80b94a3-c7d4-4b87-80b7-3d673800f7e9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:36:25.635 4076 3051113295368 MainThread DEBUG nova.compute.manager [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Refreshing instance network info cache due to event network-changed-b80b94a3-c7d4-4b87-80b7-3d673800f7e9. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:36:25.693 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:36:26.019 4076 3051113162360 GreenThread-131 DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Updating instance_info_cache with network_info: [{"id": "b80b94a3-c7d4-4b87-80b7-3d673800f7e9", "address": "fa:16:3e:77:5d:f2", "network": {"id": "6ca31cfc-0c60-4a2d-896c-cbb85a27e909", "bridge": null, "label": "tempest-network-smoke--450514377", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb80b94a3-c7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:36:26.035 4076 3051113162360 GreenThread-131 DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Releasing lock "refresh_cache-749d3e34-a2bc-4962-a645-dbcbfc91b72f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:36:26.035 4076 3051113162360 GreenThread-131 DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Instance network_info: |[{"id": "b80b94a3-c7d4-4b87-80b7-3d673800f7e9", "address": "fa:16:3e:77:5d:f2", "network": {"id": "6ca31cfc-0c60-4a2d-896c-cbb85a27e909", "bridge": null, "label": "tempest-network-smoke--450514377", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb80b94a3-c7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:36:26.035 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Preparing to wait for external event network-vif-plugged-b80b94a3-c7d4-4b87-80b7-3d673800f7e9 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:36:26.035 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:26.035 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:26.035 4076 3051113295016 MainThread DEBUG os_win.utils.compute.vmutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating VM instance-0000008f create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:36:26.044 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-749d3e34-a2bc-4962-a645-dbcbfc91b72f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:36:26.044 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Refreshing network info cache for port b80b94a3-c7d4-4b87-80b7-3d673800f7e9 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:36:26.565 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Updated VIF entry in instance network info cache for port b80b94a3-c7d4-4b87-80b7-3d673800f7e9. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:36:26.565 4076 3051113295368 MainThread DEBUG nova.network.neutron [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Updating instance_info_cache with network_info: [{"id": "b80b94a3-c7d4-4b87-80b7-3d673800f7e9", "address": "fa:16:3e:77:5d:f2", "network": {"id": "6ca31cfc-0c60-4a2d-896c-cbb85a27e909", "bridge": null, "label": "tempest-network-smoke--450514377", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapb80b94a3-c7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:36:26.585 4076 3051113295368 MainThread DEBUG oslo_concurrency.lockutils [req-831521be-794f-445a-9f87-a858566874cb 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-749d3e34-a2bc-4962-a645-dbcbfc91b72f" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:36:26.908 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:36:30.566 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-16004937-d1c3-49a8-b473-eb1ab3b8e073 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Received event network-vif-plugged-b80b94a3-c7d4-4b87-80b7-3d673800f7e9 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:36:30.566 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-16004937-d1c3-49a8-b473-eb1ab3b8e073 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:30.566 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-16004937-d1c3-49a8-b473-eb1ab3b8e073 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:30.566 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-16004937-d1c3-49a8-b473-eb1ab3b8e073 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Processing event network-vif-plugged-b80b94a3-c7d4-4b87-80b7-3d673800f7e9 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:36:30.598 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Using config drive for instance 2021-09-10 17:36:30.684 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Creating config drive at C:\openstack\instances\instance-0000008f\configdrive.iso 2021-09-10 17:36:30.737 4076 3051113295016 MainThread DEBUG oslo_concurrency.processutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp40ftvkje execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:36:31.021 4076 3051113295016 MainThread DEBUG oslo_concurrency.processutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-0000008f\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp40ftvkje" returned: 0 in 0.282s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:36:31.426 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Setting boot order for instance: instance-0000008f: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:36:31.471 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:36:31.489 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'F24A324E-22AB-4060-B65F-2B0B278806BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14846), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:31.630 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F24A324E-22AB-4060-B65F-2B0B278806BC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115946), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 31), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 31), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:31.630 4076 3051113295016 MainThread DEBUG os_win._utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.14887642860412598 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:36:31.650 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:31.650 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:31.650 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:31.650 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:31.676 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b5f89d93-c574-4704-910c-d71ee77a5224 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:31.676 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-b5f89d93-c574-4704-910c-d71ee77a5224 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:31.711 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-b5f89d93-c574-4704-910c-d71ee77a5224 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:31.746 4076 3051113162360 MainThread INFO nova.compute.manager [req-b5f89d93-c574-4704-910c-d71ee77a5224 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:32.654 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '9E6D766E-4CF7-405D-88FC-053B63D62751', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14585), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:32.788 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:32.788 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:32.788 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:32.788 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:32.801 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9E6D766E-4CF7-405D-88FC-053B63D62751', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112253), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 32), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 32), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:32.801 4076 3051113295016 MainThread DEBUG os_win._utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3201892375946045 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:36:32.801 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-0eaeef37-e228-4c70-ad66-8e0743078534 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:32.801 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-0eaeef37-e228-4c70-ad66-8e0743078534 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:32.864 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-0eaeef37-e228-4c70-ad66-8e0743078534 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:32.894 4076 3051113162360 MainThread INFO nova.compute.manager [req-0eaeef37-e228-4c70-ad66-8e0743078534 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:33.835 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '0728C02C-E535-4EEB-A587-F8C20FD7F79C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15021), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:33.959 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:33.959 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:33.987 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:33.987 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:33.987 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0728C02C-E535-4EEB-A587-F8C20FD7F79C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 33), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 33), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:33.987 4076 3051113295016 MainThread DEBUG os_win._utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5061802864074707 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:36:34.004 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-e381f25c-3f0a-4d7e-9f4b-c6e25961423f - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:34.004 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-e381f25c-3f0a-4d7e-9f4b-c6e25961423f - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:34.039 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-e381f25c-3f0a-4d7e-9f4b-c6e25961423f - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:34.058 4076 3051113162360 MainThread INFO nova.compute.manager [req-e381f25c-3f0a-4d7e-9f4b-c6e25961423f - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:35.018 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'CC44D15E-5985-449C-970E-7D316A556935', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:35.152 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:35.152 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:35.152 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:35.152 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:35.167 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CC44D15E-5985-449C-970E-7D316A556935', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120619), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 34), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 35), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:35.167 4076 3051113295016 MainThread DEBUG os_win._utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.685967206954956 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:36:35.167 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-2be209ab-2c49-4010-8a98-575c3b2450f4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:35.167 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-2be209ab-2c49-4010-8a98-575c3b2450f4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:35.220 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-2be209ab-2c49-4010-8a98-575c3b2450f4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:35.247 4076 3051113162360 MainThread INFO nova.compute.manager [req-2be209ab-2c49-4010-8a98-575c3b2450f4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:36.192 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '07611ED1-CD5E-4BEC-B0C5-B71994277663', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15363), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:36.327 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:36.327 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:36.327 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:36.327 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:36.327 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '07611ED1-CD5E-4BEC-B0C5-B71994277663', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112365), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 36), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 36), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:36.327 4076 3051113295016 MainThread DEBUG os_win._utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.846380949020386 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:36:36.351 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c540f980-dc3f-4a92-b503-d10f94eb7067 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:36.351 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-c540f980-dc3f-4a92-b503-d10f94eb7067 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:36.383 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-c540f980-dc3f-4a92-b503-d10f94eb7067 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:36.413 4076 3051113162360 MainThread INFO nova.compute.manager [req-c540f980-dc3f-4a92-b503-d10f94eb7067 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:37.359 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '619564EA-92BE-46B9-962F-169ABB8DCBF3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14833), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:37.478 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:37.478 4076 3051113293960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:37.478 4076 3051113162360 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:36:37.478 4076 3051113162360 MainThread INFO nova.compute.manager [-] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] VM Stopped (Lifecycle Event) 2021-09-10 17:36:37.501 4076 3051113295016 MainThread ERROR os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-0000008f\' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-0000008f\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)instance-0000008f77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '619564EA-92BE-46B9-962F-169ABB8DCBF3', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116623), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 36, 37), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 36, 37), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\nNot enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)\n\n'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7)", 'ErrorSummaryDescription': "'instance-0000008f' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-0000008f.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:36:37.519 4076 3051113295016 MainThread ERROR nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Failed to change vm state of instance-0000008f to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. 2021-09-10 17:36:37.519 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Got request to destroy instance 2021-09-10 17:36:37.519 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-62eae416-d57e-4916-84a6-bdfcb6d998c4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:36:37.519 4076 3051113162360 MainThread DEBUG nova.virt.hyperv.vmops [req-62eae416-d57e-4916-84a6-bdfcb6d998c4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:36:37.555 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-62eae416-d57e-4916-84a6-bdfcb6d998c4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:36:37.581 4076 3051113162360 MainThread INFO nova.compute.manager [req-62eae416-d57e-4916-84a6-bdfcb6d998c4 - - - - -] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:36:39.317 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:36:39.317 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:39.317 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:39.352 4076 3051113295016 MainThread DEBUG os_win.utils.compute.vmutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed vm state of instance-0000008f to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:36:39.352 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed state of VM instance-0000008f to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:36:39.410 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '9707017F-610F-4660-9EC1-02D8675B7224', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13801), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:36:39.520 4076 3051113295016 MainThread DEBUG os_win.utils.jobutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job succeeded: {'InstanceID': '9707017F-610F-4660-9EC1-02D8675B7224', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28204), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:36:39.520 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:39.520 4076 3051113295016 MainThread DEBUG os_win.utils.pathutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Removing directory: C:\openstack\instances\instance-0000008f check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:36:39.520 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:39.520 4076 3051113295016 MainThread ERROR nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Traceback (most recent call last): 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(*args, **kwargs) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] admin_password, network_info, block_device_info) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(self, *args, **kwds) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] should_plug_vifs=False) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'req_state': req_state}) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] time.sleep(sleep_time) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return f(*args, **kwargs) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return self._wait_for_job(job_path) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] error_desc=err_desc) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During handling of the above exception, another exception occurred: 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Traceback (most recent call last): 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] yield resources 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] accel_info=accel_info) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(*args, **kwargs) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] admin_password, network_info, block_device_info) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(self, *args, **kwds) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] should_plug_vifs=False) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'req_state': req_state}) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] time.sleep(sleep_time) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return f(*args, **kwargs) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return self._wait_for_job(job_path) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] error_desc=err_desc) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.520 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.520 4076 3051113295016 MainThread INFO nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Terminating instance 2021-09-10 17:36:39.520 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:36:39.520 4076 3051113295016 MainThread INFO nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Got request to destroy instance 2021-09-10 17:36:39.550 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.vmops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:36:39.550 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:39.550 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-0000008f" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:39.562 4076 3051113295016 MainThread INFO nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:36:39.562 4076 3051113295016 MainThread DEBUG nova.compute.claims [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:36:39.562 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:36:39.660 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:36:39.660 4076 3051113295016 MainThread DEBUG nova.virt.hyperv.hostops [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:36:39.863 4076 3051113295016 MainThread DEBUG nova.compute.provider_tree [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.scheduler.client.report [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:36:39.895 4076 3051113295016 MainThread ERROR nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Traceback (most recent call last): 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(*args, **kwargs) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] admin_password, network_info, block_device_info) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(self, *args, **kwds) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] should_plug_vifs=False) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'req_state': req_state}) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] time.sleep(sleep_time) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return f(*args, **kwargs) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return self._wait_for_job(job_path) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] error_desc=err_desc) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] During handling of the above exception, another exception occurred: 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Traceback (most recent call last): 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] accel_info=accel_info) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(*args, **kwargs) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] admin_password, network_info, block_device_info) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return function(self, *args, **kwds) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] should_plug_vifs=False) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'req_state': req_state}) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] time.sleep(sleep_time) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self.force_reraise() 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] raise self.value 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return f(*args, **kwargs) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] return self._wait_for_job(job_path) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] error_desc=err_desc) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. 2021-09-10 17:36:39.895 4076 3051113295016 MainThread TRACE nova.compute.manager [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.compute.utils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Build of instance 749d3e34-a2bc-4962-a645-dbcbfc91b72f was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-0000008f' failed to start. Not enough memory in the system to start the virtual machine instance-0000008f. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-0000008f' failed to start. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Not enough memory in the system to start the virtual machine instance-0000008f with ram size 1024 megabytes. (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) 'instance-0000008f' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 77A7DB49-33C0-4986-B9E5-6CC3AEFF32F7) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:36:39.895 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:36:40.384 4076 3051113295016 MainThread DEBUG nova.network.neutron [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:36:40.395 4076 3051113295016 MainThread INFO nova.compute.manager [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 749d3e34-a2bc-4962-a645-dbcbfc91b72f] Took 0.50 seconds to deallocate network for instance. 2021-09-10 17:36:40.569 4076 3051113295016 MainThread INFO nova.scheduler.client.report [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Deleted allocations for instance 749d3e34-a2bc-4962-a645-dbcbfc91b72f 2021-09-10 17:36:40.597 4076 3051113295016 MainThread DEBUG oslo_concurrency.lockutils [req-7437064c-c2b9-41c3-8587-4fb423d4398f 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "749d3e34-a2bc-4962-a645-dbcbfc91b72f" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 17.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:02.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:02.753 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:37:02.831 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:37:02.831 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:37:03.692 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:37:03.692 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:37:03.692 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:37:05.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:05.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:05.753 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:37:08.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:12.751 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:12.751 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:12.925 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:12.968 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:37:13.190 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:13.190 4076 3051113292376 MainThread DEBUG nova.virt.hardware [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:37:13.190 4076 3051113292376 MainThread INFO nova.compute.claims [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Claim successful on node n-h1-808474-1 2021-09-10 17:37:13.365 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:37:13.380 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:37:13.380 4076 3051113292376 MainThread DEBUG nova.compute.provider_tree [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:37:13.409 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:37:13.440 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:37:13.440 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.hostops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:37:13.458 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.hostops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:37:13.653 4076 3051113292376 MainThread DEBUG nova.compute.provider_tree [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:37:13.671 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:37:13.671 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:13.671 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:37:13.768 4076 3051113292376 MainThread DEBUG nova.compute.utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:37:13.771 4076 3051113295016 GreenThread-132 DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:37:13.771 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:37:13.787 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:37:13.893 4076 3051113295016 GreenThread-132 DEBUG nova.policy [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '580888399edc4d7c97b4a1bce73db000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '351b14c1ba6b4a35aa1c79568c8e5c30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:37:13.941 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:37:13.957 4076 3051113292376 MainThread INFO nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Spawning new instance 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG nova.block_device [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG os_win.utils.pathutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating directory: C:\openstack\instances\instance-00000090 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:37:13.957 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000090\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:37:14.005 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Resizing VHD C:\openstack\instances\instance-00000090\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:37:14.524 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Successfully created port: d7af2088-e92b-46ff-9525-d23e7b53e5b0 _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:37:14.728 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:15.284 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Successfully updated port: d7af2088-e92b-46ff-9525-d23e7b53e5b0 _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:37:15.300 4076 3051113295016 GreenThread-132 DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Acquired lock "refresh_cache-0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:37:15.300 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:37:15.331 4076 3051119620696 MainThread DEBUG nova.compute.manager [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Received event network-changed-d7af2088-e92b-46ff-9525-d23e7b53e5b0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:37:15.331 4076 3051119620696 MainThread DEBUG nova.compute.manager [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Refreshing instance network info cache due to event network-changed-d7af2088-e92b-46ff-9525-d23e7b53e5b0. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:37:15.365 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:37:15.706 4076 3051113295016 GreenThread-132 DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Updating instance_info_cache with network_info: [{"id": "d7af2088-e92b-46ff-9525-d23e7b53e5b0", "address": "fa:16:3e:fc:38:b7", "network": {"id": "46a10a17-af3d-45de-99a2-250cdcb041b2", "bridge": null, "label": "tempest-network-smoke--396985115", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd7af2088-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:37:15.722 4076 3051113295016 GreenThread-132 DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Releasing lock "refresh_cache-0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:37:15.722 4076 3051113295016 GreenThread-132 DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Instance network_info: |[{"id": "d7af2088-e92b-46ff-9525-d23e7b53e5b0", "address": "fa:16:3e:fc:38:b7", "network": {"id": "46a10a17-af3d-45de-99a2-250cdcb041b2", "bridge": null, "label": "tempest-network-smoke--396985115", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd7af2088-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:37:15.722 4076 3051119620696 MainThread DEBUG oslo_concurrency.lockutils [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:37:15.722 4076 3051119620696 MainThread DEBUG nova.network.neutron [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Refreshing network info cache for port d7af2088-e92b-46ff-9525-d23e7b53e5b0 _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:37:15.722 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Preparing to wait for external event network-vif-plugged-d7af2088-e92b-46ff-9525-d23e7b53e5b0 prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:37:15.722 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:15.722 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:15.722 4076 3051113292376 MainThread DEBUG os_win.utils.compute.vmutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating VM instance-00000090 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:37:15.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:15.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:15.767 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:15.767 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:15.767 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:37:15.767 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:37:15.791 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:37:16.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:37:16.020 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:37:16.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:37:16.193 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:37:16.207 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:37:16.286 4076 3051119620696 MainThread DEBUG nova.network.neutron [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Updated VIF entry in instance network info cache for port d7af2088-e92b-46ff-9525-d23e7b53e5b0. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:37:16.286 4076 3051119620696 MainThread DEBUG nova.network.neutron [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Updating instance_info_cache with network_info: [{"id": "d7af2088-e92b-46ff-9525-d23e7b53e5b0", "address": "fa:16:3e:fc:38:b7", "network": {"id": "46a10a17-af3d-45de-99a2-250cdcb041b2", "bridge": null, "label": "tempest-network-smoke--396985115", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapd7af2088-e9", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:37:16.286 4076 3051119620696 MainThread DEBUG oslo_concurrency.lockutils [req-8b7eb27e-e301-4969-8b4d-2f7655e02fc4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:37:16.413 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:37:16.433 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:37:16.433 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:37:16.433 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:16.655 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:37:17.425 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:17.459 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:20.597 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-da00209d-32a4-4147-a5d2-436c7add613d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Received event network-vif-plugged-d7af2088-e92b-46ff-9525-d23e7b53e5b0 external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:37:20.597 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-da00209d-32a4-4147-a5d2-436c7add613d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:20.597 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [req-da00209d-32a4-4147-a5d2-436c7add613d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:20.612 4076 3051113162360 MainThread DEBUG nova.compute.manager [req-da00209d-32a4-4147-a5d2-436c7add613d 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Processing event network-vif-plugged-d7af2088-e92b-46ff-9525-d23e7b53e5b0 _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:37:20.628 4076 3051113292376 MainThread INFO nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Using config drive for instance 2021-09-10 17:37:20.750 4076 3051113292376 MainThread INFO nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Creating config drive at C:\openstack\instances\instance-00000090\configdrive.iso 2021-09-10 17:37:20.786 4076 3051113292376 MainThread DEBUG oslo_concurrency.processutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000090\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8jgcsz5f execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:37:21.100 4076 3051113292376 MainThread DEBUG oslo_concurrency.processutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000090\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp8jgcsz5f" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:37:21.473 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Setting boot order for instance: instance-00000090: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:37:21.519 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:37:21.566 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '5432C424-2565-4930-A5B7-1E3D9506C579', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19849), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:21.697 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:21.697 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:21.707 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:21.707 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:21.707 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5432C424-2565-4930-A5B7-1E3D9506C579', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115441), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 21), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 21), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:21.707 4076 3051113292376 MainThread DEBUG os_win._utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18789052963256836 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:37:21.707 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-444d8cea-8fa5-4d3a-bb1f-eb6635d9cfd6 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:21.707 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-444d8cea-8fa5-4d3a-bb1f-eb6635d9cfd6 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:21.762 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-444d8cea-8fa5-4d3a-bb1f-eb6635d9cfd6 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:21.784 4076 3051113162008 MainThread INFO nova.compute.manager [req-444d8cea-8fa5-4d3a-bb1f-eb6635d9cfd6 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:22.737 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '093B3ADD-EDA7-48E9-B7DD-ECD57FF52565', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15127), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:22.864 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:22.864 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:22.864 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:22.864 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:22.881 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '093B3ADD-EDA7-48E9-B7DD-ECD57FF52565', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112650), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 22), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 22), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:22.881 4076 3051113292376 MainThread DEBUG os_win._utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3613982200622559 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:37:22.881 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-ae10fbfe-92e1-4281-b363-93e6fe9eaf56 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:22.881 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-ae10fbfe-92e1-4281-b363-93e6fe9eaf56 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:22.932 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-ae10fbfe-92e1-4281-b363-93e6fe9eaf56 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:22.956 4076 3051113162008 MainThread INFO nova.compute.manager [req-ae10fbfe-92e1-4281-b363-93e6fe9eaf56 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:23.897 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'FF2E37E6-C96C-41C2-B237-D851DF4A16BB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15798), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:24.040 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:24.040 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:24.040 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:24.040 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:24.040 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'FF2E37E6-C96C-41C2-B237-D851DF4A16BB', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112825), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 23), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 24), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:24.051 4076 3051113292376 MainThread DEBUG os_win._utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.532289981842041 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:37:24.051 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-43d9db22-4b1e-4ed1-b7ca-9d29ba7aa09a - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:24.051 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-43d9db22-4b1e-4ed1-b7ca-9d29ba7aa09a - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:24.104 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-43d9db22-4b1e-4ed1-b7ca-9d29ba7aa09a - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:24.114 4076 3051113162008 MainThread INFO nova.compute.manager [req-43d9db22-4b1e-4ed1-b7ca-9d29ba7aa09a - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:24.761 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:24.761 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:37:25.083 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'BE2A5557-A879-4602-9A7C-1F727857EA56', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14916), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:25.206 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:25.206 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:25.206 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:25.206 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:25.206 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BE2A5557-A879-4602-9A7C-1F727857EA56', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116079), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 25), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 25), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:25.206 4076 3051113292376 MainThread DEBUG os_win._utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.6867122650146484 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:37:25.242 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-e831b65f-9599-4bf8-b827-d7684fb6d1e4 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:25.242 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-e831b65f-9599-4bf8-b827-d7684fb6d1e4 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:25.270 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-e831b65f-9599-4bf8-b827-d7684fb6d1e4 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:25.301 4076 3051113162008 MainThread INFO nova.compute.manager [req-e831b65f-9599-4bf8-b827-d7684fb6d1e4 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:26.239 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '822AC944-50E2-4CCA-940C-D50C71EBDBEC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14766), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:26.381 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:26.381 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:26.381 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:26.381 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:26.381 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '822AC944-50E2-4CCA-940C-D50C71EBDBEC', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111557), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 26), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 26), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:26.381 4076 3051113292376 MainThread DEBUG os_win._utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.861960172653198 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:37:26.405 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d858f82a-cc86-4292-b2f4-7e67ab3efdd5 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:26.405 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-d858f82a-cc86-4292-b2f4-7e67ab3efdd5 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:26.441 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-d858f82a-cc86-4292-b2f4-7e67ab3efdd5 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:26.473 4076 3051113162008 MainThread INFO nova.compute.manager [req-d858f82a-cc86-4292-b2f4-7e67ab3efdd5 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:27.426 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '12086A27-4A85-4A16-AD7E-7BDBA67E4003', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15107), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:27.551 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:27.551 4076 3051113162360 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:27.551 4076 3051113162008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:37:27.551 4076 3051113162008 MainThread INFO nova.compute.manager [-] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] VM Stopped (Lifecycle Event) 2021-09-10 17:37:27.551 4076 3051113292376 MainThread ERROR os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000090\' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54712030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E54710243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000090\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)instance-0000009092E96BAD-CAF4-4BB3-AC4B-12C93D42E547%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '12086A27-4A85-4A16-AD7E-7BDBA67E4003', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117940), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 37, 27), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 37, 27), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\nNot enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)\n\n'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547)", 'ErrorSummaryDescription': "'instance-00000090' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000090.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:37:27.551 4076 3051113292376 MainThread ERROR nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Failed to change vm state of instance-00000090 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. 2021-09-10 17:37:27.551 4076 3051113292376 MainThread INFO nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Got request to destroy instance 2021-09-10 17:37:27.577 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-dfef6901-b9da-4db9-a74b-d90bb0282634 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:37:27.577 4076 3051113162008 MainThread DEBUG nova.virt.hyperv.vmops [req-dfef6901-b9da-4db9-a74b-d90bb0282634 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:37:27.635 4076 3051113162008 MainThread DEBUG nova.compute.manager [req-dfef6901-b9da-4db9-a74b-d90bb0282634 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:37:27.647 4076 3051113162008 MainThread INFO nova.compute.manager [req-dfef6901-b9da-4db9-a74b-d90bb0282634 - - - - -] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:37:29.367 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:37:29.367 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:29.367 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:29.396 4076 3051113292376 MainThread DEBUG os_win.utils.compute.vmutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed vm state of instance-00000090 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:37:29.396 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed state of VM instance-00000090 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:37:29.432 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'DC5B8914-EE4A-4F8D-A319-71C4F68C039F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=13951), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:37:29.536 4076 3051113292376 MainThread DEBUG os_win.utils.jobutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job succeeded: {'InstanceID': 'DC5B8914-EE4A-4F8D-A319-71C4F68C039F', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28868), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:37:29.536 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:29.536 4076 3051113292376 MainThread DEBUG os_win.utils.pathutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Removing directory: C:\openstack\instances\instance-00000090 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:37:29.536 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:29.536 4076 3051113292376 MainThread ERROR nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Traceback (most recent call last): 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(*args, **kwargs) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] admin_password, network_info, block_device_info) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(self, *args, **kwds) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] should_plug_vifs=False) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'req_state': req_state}) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] time.sleep(sleep_time) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return f(*args, **kwargs) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return self._wait_for_job(job_path) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] error_desc=err_desc) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During handling of the above exception, another exception occurred: 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Traceback (most recent call last): 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] yield resources 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] accel_info=accel_info) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(*args, **kwargs) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] admin_password, network_info, block_device_info) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(self, *args, **kwds) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] should_plug_vifs=False) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'req_state': req_state}) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] time.sleep(sleep_time) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return f(*args, **kwargs) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return self._wait_for_job(job_path) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] error_desc=err_desc) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.536 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.536 4076 3051113292376 MainThread INFO nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Terminating instance 2021-09-10 17:37:29.536 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:37:29.536 4076 3051113292376 MainThread INFO nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Got request to destroy instance 2021-09-10 17:37:29.565 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.vmops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:37:29.565 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:29.565 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000090" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:29.581 4076 3051113292376 MainThread INFO nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Took 0.05 seconds to destroy the instance on the hypervisor. 2021-09-10 17:37:29.583 4076 3051113292376 MainThread DEBUG nova.compute.claims [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:37:29.583 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:37:29.659 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.hostops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:37:29.659 4076 3051113292376 MainThread DEBUG nova.virt.hyperv.hostops [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:37:29.877 4076 3051113292376 MainThread DEBUG nova.compute.provider_tree [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:37:29.889 4076 3051113292376 MainThread DEBUG nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:37:29.889 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.313s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:29.889 4076 3051113292376 MainThread ERROR nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Traceback (most recent call last): 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(*args, **kwargs) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] admin_password, network_info, block_device_info) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(self, *args, **kwds) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] should_plug_vifs=False) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'req_state': req_state}) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] time.sleep(sleep_time) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return f(*args, **kwargs) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return self._wait_for_job(job_path) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] error_desc=err_desc) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] During handling of the above exception, another exception occurred: 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Traceback (most recent call last): 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] accel_info=accel_info) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(*args, **kwargs) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] admin_password, network_info, block_device_info) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return function(self, *args, **kwds) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] should_plug_vifs=False) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'req_state': req_state}) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] time.sleep(sleep_time) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self.force_reraise() 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] raise self.value 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return f(*args, **kwargs) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] return self._wait_for_job(job_path) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] error_desc=err_desc) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. 2021-09-10 17:37:29.889 4076 3051113292376 MainThread TRACE nova.compute.manager [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] 2021-09-10 17:37:29.889 4076 3051113292376 MainThread DEBUG nova.compute.utils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:37:29.909 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Build of instance 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000090' failed to start. Not enough memory in the system to start the virtual machine instance-00000090. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000090' failed to start. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Not enough memory in the system to start the virtual machine instance-00000090 with ram size 1024 megabytes. (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) 'instance-00000090' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 92E96BAD-CAF4-4BB3-AC4B-12C93D42E547) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:37:29.909 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:37:29.909 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:37:29.909 4076 3051113292376 MainThread DEBUG nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:37:29.913 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:37:30.447 4076 3051113292376 MainThread DEBUG nova.network.neutron [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:37:30.473 4076 3051113292376 MainThread INFO nova.compute.manager [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae] Took 0.56 seconds to deallocate network for instance. 2021-09-10 17:37:30.645 4076 3051113292376 MainThread INFO nova.scheduler.client.report [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Deleted allocations for instance 0e545b2c-ea5d-4c5f-a9fd-35267850e2ae 2021-09-10 17:37:30.659 4076 3051113292376 MainThread DEBUG oslo_concurrency.lockutils [req-ed9a90b6-b473-4cd2-8ca0-bb172ef43b94 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "0e545b2c-ea5d-4c5f-a9fd-35267850e2ae" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 17.734s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:37:45.783 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:37:45.783 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:37:45.787 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:38:01.771 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "607bf05f-387c-4106-872a-1b453d094222" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:01.792 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:38:01.880 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:01.895 4076 3051113164120 MainThread DEBUG nova.virt.hardware [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:38:01.895 4076 3051113164120 MainThread INFO nova.compute.claims [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Claim successful on node n-h1-808474-1 2021-09-10 17:38:01.989 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:38:02.021 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:38:02.224 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:38:02.236 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:38:02.236 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:02.236 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:38:02.318 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:38:02.318 4076 3051113162536 GreenThread-133 DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:38:02.318 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:38:02.339 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:38:02.494 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:38:02.494 4076 3051113162536 GreenThread-133 DEBUG nova.policy [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '580888399edc4d7c97b4a1bce73db000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '351b14c1ba6b4a35aa1c79568c8e5c30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:38:02.505 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Spawning new instance 2021-09-10 17:38:02.505 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:02.505 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:02.521 4076 3051113164120 MainThread DEBUG nova.block_device [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:38:02.521 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:02.521 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:02.521 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating directory: C:\openstack\instances\instance-00000091 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:38:02.521 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000091\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:38:02.533 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Resizing VHD C:\openstack\instances\instance-00000091\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:38:03.161 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Successfully created port: 98100f3e-8d02-44b6-963b-6a4bbe5e48ca _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:38:03.901 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Successfully updated port: 98100f3e-8d02-44b6-963b-6a4bbe5e48ca _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:38:03.910 4076 3051113162536 GreenThread-133 DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Acquired lock "refresh_cache-607bf05f-387c-4106-872a-1b453d094222" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:38:03.910 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:38:03.958 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Received event network-changed-98100f3e-8d02-44b6-963b-6a4bbe5e48ca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:38:03.958 4076 3051113164472 MainThread DEBUG nova.compute.manager [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Refreshing instance network info cache due to event network-changed-98100f3e-8d02-44b6-963b-6a4bbe5e48ca. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:38:04.002 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:38:04.318 4076 3051113162536 GreenThread-133 DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Updating instance_info_cache with network_info: [{"id": "98100f3e-8d02-44b6-963b-6a4bbe5e48ca", "address": "fa:16:3e:17:16:d1", "network": {"id": "8ff1723c-2a3d-4249-8dcc-dd7eb3de6601", "bridge": null, "label": "tempest-network-smoke--1013058763", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap98100f3e-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:04.346 4076 3051113162536 GreenThread-133 DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Releasing lock "refresh_cache-607bf05f-387c-4106-872a-1b453d094222" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:38:04.346 4076 3051113162536 GreenThread-133 DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Instance network_info: |[{"id": "98100f3e-8d02-44b6-963b-6a4bbe5e48ca", "address": "fa:16:3e:17:16:d1", "network": {"id": "8ff1723c-2a3d-4249-8dcc-dd7eb3de6601", "bridge": null, "label": "tempest-network-smoke--1013058763", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap98100f3e-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:38:04.346 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Preparing to wait for external event network-vif-plugged-98100f3e-8d02-44b6-963b-6a4bbe5e48ca prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:38:04.346 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "607bf05f-387c-4106-872a-1b453d094222-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:04.346 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "607bf05f-387c-4106-872a-1b453d094222-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:04.346 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating VM instance-00000091 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:38:04.346 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-607bf05f-387c-4106-872a-1b453d094222" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:38:04.346 4076 3051113164472 MainThread DEBUG nova.network.neutron [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Refreshing network info cache for port 98100f3e-8d02-44b6-963b-6a4bbe5e48ca _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:38:04.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:04.747 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:38:04.827 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:38:04.827 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:38:04.880 4076 3051113164472 MainThread DEBUG nova.network.neutron [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Updated VIF entry in instance network info cache for port 98100f3e-8d02-44b6-963b-6a4bbe5e48ca. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:38:04.880 4076 3051113164472 MainThread DEBUG nova.network.neutron [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Updating instance_info_cache with network_info: [{"id": "98100f3e-8d02-44b6-963b-6a4bbe5e48ca", "address": "fa:16:3e:17:16:d1", "network": {"id": "8ff1723c-2a3d-4249-8dcc-dd7eb3de6601", "bridge": null, "label": "tempest-network-smoke--1013058763", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap98100f3e-8d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:04.895 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [req-5ce14468-153c-4abf-8742-6915ea665a37 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-607bf05f-387c-4106-872a-1b453d094222" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:38:05.231 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:38:05.334 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:05.351 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:38:05.351 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:38:05.746 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:05.746 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:05.746 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:38:08.818 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-3adebf02-a649-451e-8df7-e3b79b95c8a9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Received event network-vif-plugged-98100f3e-8d02-44b6-963b-6a4bbe5e48ca external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:38:08.818 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-3adebf02-a649-451e-8df7-e3b79b95c8a9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "607bf05f-387c-4106-872a-1b453d094222-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:08.818 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [req-3adebf02-a649-451e-8df7-e3b79b95c8a9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "607bf05f-387c-4106-872a-1b453d094222-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:08.818 4076 3051113162536 MainThread DEBUG nova.compute.manager [req-3adebf02-a649-451e-8df7-e3b79b95c8a9 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Processing event network-vif-plugged-98100f3e-8d02-44b6-963b-6a4bbe5e48ca _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:38:08.865 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Using config drive for instance 2021-09-10 17:38:08.970 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Creating config drive at C:\openstack\instances\instance-00000091\configdrive.iso 2021-09-10 17:38:09.003 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000091\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy8iamvt8 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:38:09.354 4076 3051113164120 MainThread DEBUG oslo_concurrency.processutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000091\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpy8iamvt8" returned: 0 in 0.359s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:38:09.725 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Setting boot order for instance: instance-00000091: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:38:09.768 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:38:09.809 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '16FB8AF8-784E-4DB9-AB16-C62BB48711D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14374), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:09.942 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:09.942 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:09.942 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:09.942 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:09.959 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '16FB8AF8-784E-4DB9-AB16-C62BB48711D6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=115817), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 9), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 9), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:09.959 4076 3051113164120 MainThread DEBUG os_win._utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19114112854003906 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:38:09.959 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-efddd5de-d90d-4a15-9ad1-7454338301e3 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:09.959 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-efddd5de-d90d-4a15-9ad1-7454338301e3 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:10.005 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-efddd5de-d90d-4a15-9ad1-7454338301e3 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:10.030 4076 3051113163416 MainThread INFO nova.compute.manager [req-efddd5de-d90d-4a15-9ad1-7454338301e3 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:10.756 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:10.962 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '12CCE3BA-8674-4BAC-A394-86BDF82E692C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15284), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:11.111 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:11.111 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:11.111 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:11.111 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:11.116 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '12CCE3BA-8674-4BAC-A394-86BDF82E692C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110813), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 10), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 11), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:11.116 4076 3051113164120 MainThread DEBUG os_win._utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3480021953582764 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:38:11.128 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-762ccdd1-7e07-4215-a333-cdad36cff5a2 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:11.128 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-762ccdd1-7e07-4215-a333-cdad36cff5a2 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:11.181 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-762ccdd1-7e07-4215-a333-cdad36cff5a2 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:11.205 4076 3051113163416 MainThread INFO nova.compute.manager [req-762ccdd1-7e07-4215-a333-cdad36cff5a2 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:11.949 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:12.145 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '0C9B079F-027D-4436-8C7D-04E142C5782E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=21241), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:12.252 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:12.252 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:12.252 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:12.283 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.031s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:12.283 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '0C9B079F-027D-4436-8C7D-04E142C5782E', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116124), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 12), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 12), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:12.283 4076 3051113164120 MainThread DEBUG os_win._utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.514893054962158 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:38:12.298 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-99073efb-1644-4c27-aa45-3bbe9b3ade30 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:12.298 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-99073efb-1644-4c27-aa45-3bbe9b3ade30 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:12.333 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-99073efb-1644-4c27-aa45-3bbe9b3ade30 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:12.374 4076 3051113163416 MainThread INFO nova.compute.manager [req-99073efb-1644-4c27-aa45-3bbe9b3ade30 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:13.324 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '654E1B7C-837B-4FCC-B85B-4AF19B4128F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14298), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:13.456 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:13.456 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:13.456 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:13.456 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:13.474 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-5bc2b30f-b197-410f-bfff-138adf1c0fd1 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:13.474 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-5bc2b30f-b197-410f-bfff-138adf1c0fd1 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:13.474 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '654E1B7C-837B-4FCC-B85B-4AF19B4128F4', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111569), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 13), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 13), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:13.474 4076 3051113164120 MainThread DEBUG os_win._utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.705747127532959 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:38:13.536 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-5bc2b30f-b197-410f-bfff-138adf1c0fd1 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:13.536 4076 3051113163416 MainThread INFO nova.compute.manager [req-5bc2b30f-b197-410f-bfff-138adf1c0fd1 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:14.489 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'F25E4BB1-984D-4874-88BB-F1D588628520', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15103), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:14.630 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:14.630 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:14.630 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:14.630 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:14.630 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F25E4BB1-984D-4874-88BB-F1D588628520', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111706), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 14), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:14.630 4076 3051113164120 MainThread DEBUG os_win._utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.86184549331665 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:38:14.658 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-9bce71e6-7683-486d-ad67-8bed4420507d - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:14.658 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-9bce71e6-7683-486d-ad67-8bed4420507d - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:14.690 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-9bce71e6-7683-486d-ad67-8bed4420507d - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:14.727 4076 3051113163416 MainThread INFO nova.compute.manager [req-9bce71e6-7683-486d-ad67-8bed4420507d - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:14.750 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:14.750 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:15.646 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '9CFAAC14-4003-4B3B-AFC4-6A7E1A09CFED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14734), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:15.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:15.763 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:15.776 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:15.776 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:15.776 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:38:15.776 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:38:15.786 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:15.786 4076 3051113162536 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:15.786 4076 3051113163416 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:38:15.786 4076 3051113163416 MainThread INFO nova.compute.manager [-] [instance: 607bf05f-387c-4106-872a-1b453d094222] VM Stopped (Lifecycle Event) 2021-09-10 17:38:15.799 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:38:15.799 4076 3051113164120 MainThread ERROR os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000091\' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E812030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E810243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000091\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)instance-00000091663215E4-24D2-471E-9038-4F14AD12B4E8%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9CFAAC14-4003-4B3B-AFC4-6A7E1A09CFED', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=110918), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 15), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\nNot enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)\n\n'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8)", 'ErrorSummaryDescription': "'instance-00000091' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000091.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:38:15.799 4076 3051113164120 MainThread ERROR nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Failed to change vm state of instance-00000091 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. 2021-09-10 17:38:15.799 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Got request to destroy instance 2021-09-10 17:38:15.814 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-30169928-84cf-44ff-9d8f-211450a73310 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:38:15.814 4076 3051113163416 MainThread DEBUG nova.virt.hyperv.vmops [req-30169928-84cf-44ff-9d8f-211450a73310 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:15.862 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-30169928-84cf-44ff-9d8f-211450a73310 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:38:15.880 4076 3051113163416 MainThread INFO nova.compute.manager [req-30169928-84cf-44ff-9d8f-211450a73310 - - - - -] [instance: 607bf05f-387c-4106-872a-1b453d094222] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:38:16.010 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2198MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:38:16.010 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:16.130 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.130 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 607bf05f-387c-4106-872a-1b453d094222 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:38:16.158 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:38:16.177 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:38:16.205 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:38:16.426 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:38:16.426 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:38:16.426 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:38:16.449 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.453s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:17.423 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:17.658 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:38:17.658 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:17.658 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:17.677 4076 3051113164120 MainThread DEBUG os_win.utils.compute.vmutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed vm state of instance-00000091 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:38:17.677 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed state of VM instance-00000091 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:38:17.738 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '9CA1B035-5A05-4A0A-AA4A-E6F29E4D1683', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14699), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:38:17.848 4076 3051113164120 MainThread DEBUG os_win.utils.jobutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job succeeded: {'InstanceID': '9CA1B035-5A05-4A0A-AA4A-E6F29E4D1683', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30077), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:38:17.848 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:17.848 4076 3051113164120 MainThread DEBUG os_win.utils.pathutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Removing directory: C:\openstack\instances\instance-00000091 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:38:17.848 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:17.848 4076 3051113164120 MainThread ERROR nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Traceback (most recent call last): 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(*args, **kwargs) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] admin_password, network_info, block_device_info) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(self, *args, **kwds) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] should_plug_vifs=False) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'req_state': req_state}) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] time.sleep(sleep_time) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return f(*args, **kwargs) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return self._wait_for_job(job_path) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] error_desc=err_desc) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] During handling of the above exception, another exception occurred: 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Traceback (most recent call last): 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] yield resources 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] accel_info=accel_info) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(*args, **kwargs) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] admin_password, network_info, block_device_info) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(self, *args, **kwds) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] should_plug_vifs=False) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'req_state': req_state}) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] time.sleep(sleep_time) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return f(*args, **kwargs) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return self._wait_for_job(job_path) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] error_desc=err_desc) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:17.848 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:17.871 4076 3051113164120 MainThread INFO nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Terminating instance 2021-09-10 17:38:17.871 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:38:17.871 4076 3051113164120 MainThread INFO nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Got request to destroy instance 2021-09-10 17:38:17.879 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:38:17.879 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:17.879 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000091" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:17.879 4076 3051113164120 MainThread INFO nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Took 0.00 seconds to destroy the instance on the hypervisor. 2021-09-10 17:38:17.895 4076 3051113164120 MainThread DEBUG nova.compute.claims [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:38:17.895 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:17.986 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:38:18.002 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.hostops [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:38:18.174 4076 3051113164120 MainThread DEBUG nova.compute.provider_tree [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.scheduler.client.report [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.312s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:18.208 4076 3051113164120 MainThread ERROR nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Traceback (most recent call last): 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(*args, **kwargs) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] admin_password, network_info, block_device_info) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(self, *args, **kwds) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] should_plug_vifs=False) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'req_state': req_state}) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] time.sleep(sleep_time) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return f(*args, **kwargs) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return self._wait_for_job(job_path) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] error_desc=err_desc) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] During handling of the above exception, another exception occurred: 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Traceback (most recent call last): 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] accel_info=accel_info) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(*args, **kwargs) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] admin_password, network_info, block_device_info) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return function(self, *args, **kwds) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] should_plug_vifs=False) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'req_state': req_state}) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] time.sleep(sleep_time) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self.force_reraise() 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] raise self.value 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return f(*args, **kwargs) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] return self._wait_for_job(job_path) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] error_desc=err_desc) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. 2021-09-10 17:38:18.208 4076 3051113164120 MainThread TRACE nova.compute.manager [instance: 607bf05f-387c-4106-872a-1b453d094222] 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.compute.utils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Build of instance 607bf05f-387c-4106-872a-1b453d094222 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000091' failed to start. Not enough memory in the system to start the virtual machine instance-00000091. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000091' failed to start. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Not enough memory in the system to start the virtual machine instance-00000091 with ram size 1024 megabytes. (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) 'instance-00000091' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 663215E4-24D2-471E-9038-4F14AD12B4E8) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:38:18.208 4076 3051113164120 MainThread DEBUG nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:38:18.228 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:38:18.709 4076 3051113164120 MainThread DEBUG nova.network.neutron [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:18.733 4076 3051113164120 MainThread INFO nova.compute.manager [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 607bf05f-387c-4106-872a-1b453d094222] Took 0.51 seconds to deallocate network for instance. 2021-09-10 17:38:18.899 4076 3051113164120 MainThread INFO nova.scheduler.client.report [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Deleted allocations for instance 607bf05f-387c-4106-872a-1b453d094222 2021-09-10 17:38:18.927 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [req-c79823fc-2720-4001-baf5-254ee329aa98 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "607bf05f-387c-4106-872a-1b453d094222" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 17.156s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:50.177 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:50.196 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:38:50.298 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:50.298 4076 3051113163944 MainThread DEBUG nova.virt.hardware [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:38:50.298 4076 3051113163944 MainThread INFO nova.compute.claims [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Claim successful on node n-h1-808474-1 2021-09-10 17:38:50.411 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:38:50.411 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:38:50.644 4076 3051113163944 MainThread DEBUG nova.compute.provider_tree [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:38:50.644 4076 3051113163944 MainThread DEBUG nova.scheduler.client.report [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:38:50.673 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:50.673 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:38:50.735 4076 3051113163944 MainThread DEBUG nova.compute.utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:38:50.735 4076 3051113163416 GreenThread-134 DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:38:50.735 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:38:50.755 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:38:50.846 4076 3051113163416 GreenThread-134 DEBUG nova.policy [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '580888399edc4d7c97b4a1bce73db000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '351b14c1ba6b4a35aa1c79568c8e5c30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:38:50.907 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:38:50.925 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Spawning new instance 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG nova.block_device [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:50.925 4076 3051113163944 MainThread DEBUG os_win.utils.pathutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating directory: C:\openstack\instances\instance-00000092 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:38:50.954 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000092\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:38:50.954 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:38:50.970 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Resizing VHD C:\openstack\instances\instance-00000092\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 WARNING nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] While synchronizing instance power states, found 13 instances in the database and 12 instances on the hypervisor. 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:38:51.033 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.033 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.033 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.033 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.033 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.033 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113162888 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.048 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:51.048 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:38:51.174 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.141s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.189 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.157s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.235 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.187s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.249 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.255 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.219s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.255 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.255 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.255 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.291 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.291 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.291 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.291 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.250s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:51.567 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Successfully created port: 2e70a2d6-eb63-45da-857e-184b3ffccc0f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:38:52.517 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Successfully updated port: 2e70a2d6-eb63-45da-857e-184b3ffccc0f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:38:52.517 4076 3051113163416 GreenThread-134 DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Acquired lock "refresh_cache-436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:38:52.517 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:38:52.601 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Received event network-changed-2e70a2d6-eb63-45da-857e-184b3ffccc0f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:38:52.601 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Refreshing instance network info cache due to event network-changed-2e70a2d6-eb63-45da-857e-184b3ffccc0f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:38:52.601 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:38:52.970 4076 3051113163416 GreenThread-134 DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Updating instance_info_cache with network_info: [{"id": "2e70a2d6-eb63-45da-857e-184b3ffccc0f", "address": "fa:16:3e:9f:e8:60", "network": {"id": "d66ed97b-17c5-4a50-aa11-82413968de7e", "bridge": null, "label": "tempest-network-smoke--272333549", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2e70a2d6-eb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:52.989 4076 3051113163416 GreenThread-134 DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Releasing lock "refresh_cache-436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:38:52.989 4076 3051113163416 GreenThread-134 DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Instance network_info: |[{"id": "2e70a2d6-eb63-45da-857e-184b3ffccc0f", "address": "fa:16:3e:9f:e8:60", "network": {"id": "d66ed97b-17c5-4a50-aa11-82413968de7e", "bridge": null, "label": "tempest-network-smoke--272333549", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2e70a2d6-eb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:38:52.989 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:38:52.989 4076 3051055215720 MainThread DEBUG nova.network.neutron [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Refreshing network info cache for port 2e70a2d6-eb63-45da-857e-184b3ffccc0f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:38:53.002 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Preparing to wait for external event network-vif-plugged-2e70a2d6-eb63-45da-857e-184b3ffccc0f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:38:53.002 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:53.002 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:53.002 4076 3051113163944 MainThread DEBUG os_win.utils.compute.vmutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Creating VM instance-00000092 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:38:53.496 4076 3051055215720 MainThread DEBUG nova.network.neutron [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Updated VIF entry in instance network info cache for port 2e70a2d6-eb63-45da-857e-184b3ffccc0f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:38:53.496 4076 3051055215720 MainThread DEBUG nova.network.neutron [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Updating instance_info_cache with network_info: [{"id": "2e70a2d6-eb63-45da-857e-184b3ffccc0f", "address": "fa:16:3e:9f:e8:60", "network": {"id": "d66ed97b-17c5-4a50-aa11-82413968de7e", "bridge": null, "label": "tempest-network-smoke--272333549", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "351b14c1ba6b4a35aa1c79568c8e5c30", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap2e70a2d6-eb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:38:53.518 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-67d0b9c9-892a-4aef-8db7-28e3d2f7e455 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:38:53.910 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:38:58.955 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-a39e477f-1aaf-4711-b902-2c13cbd59a3e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Received event network-vif-plugged-2e70a2d6-eb63-45da-857e-184b3ffccc0f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:38:58.955 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-a39e477f-1aaf-4711-b902-2c13cbd59a3e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:38:58.955 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [req-a39e477f-1aaf-4711-b902-2c13cbd59a3e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:38:58.955 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-a39e477f-1aaf-4711-b902-2c13cbd59a3e 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Processing event network-vif-plugged-2e70a2d6-eb63-45da-857e-184b3ffccc0f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:38:58.989 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Using config drive for instance 2021-09-10 17:38:59.101 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Creating config drive at C:\openstack\instances\instance-00000092\configdrive.iso 2021-09-10 17:38:59.130 4076 3051113163944 MainThread DEBUG oslo_concurrency.processutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000092\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_bgkaoku execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:38:59.438 4076 3051113163944 MainThread DEBUG oslo_concurrency.processutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000092\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp_bgkaoku" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:38:59.805 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Setting boot order for instance: instance-00000092: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:38:59.866 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:38:59.911 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '4CF5B6E8-599F-43F1-AC5A-24F10DC71664', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18948), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:00.040 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:00.040 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:00.050 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:00.050 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:00.050 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '4CF5B6E8-599F-43F1-AC5A-24F10DC71664', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=116735), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 38, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 38, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:00.050 4076 3051113163944 MainThread DEBUG os_win._utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1840224266052246 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:00.065 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-9822561a-9787-40a6-a046-60e7fee4df52 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:00.065 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-9822561a-9787-40a6-a046-60e7fee4df52 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:00.100 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-9822561a-9787-40a6-a046-60e7fee4df52 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:00.130 4076 3051055214664 MainThread INFO nova.compute.manager [req-9822561a-9787-40a6-a046-60e7fee4df52 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:01.083 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '5AF07B0D-1CF4-40AA-ACD2-A7CE619F8799', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15403), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:01.204 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:01.204 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:01.204 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:01.204 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:01.233 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5AF07B0D-1CF4-40AA-ACD2-A7CE619F8799', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114899), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:01.233 4076 3051113163944 MainThread DEBUG os_win._utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3673577308654785 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:01.246 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c56cb0b9-f3c5-4972-a214-e09a632a7331 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:01.246 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-c56cb0b9-f3c5-4972-a214-e09a632a7331 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:01.282 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-c56cb0b9-f3c5-4972-a214-e09a632a7331 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:01.303 4076 3051055214664 MainThread INFO nova.compute.manager [req-c56cb0b9-f3c5-4972-a214-e09a632a7331 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:02.271 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '9DD028AA-FD6B-4F47-8D31-9D9C389C71B7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15550), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:02.407 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:02.407 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:02.407 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:02.407 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:02.423 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '9DD028AA-FD6B-4F47-8D31-9D9C389C71B7', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113031), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:02.423 4076 3051113163944 MainThread DEBUG os_win._utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5572121143341064 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:02.423 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-a4b05e8f-6c41-41d5-9e5c-358075eaefdc - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:02.423 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-a4b05e8f-6c41-41d5-9e5c-358075eaefdc - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:02.489 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-a4b05e8f-6c41-41d5-9e5c-358075eaefdc - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:02.505 4076 3051055214664 MainThread INFO nova.compute.manager [req-a4b05e8f-6c41-41d5-9e5c-358075eaefdc - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:03.453 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '58F602D7-2404-4234-8211-737FD1E360E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14806), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:03.581 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:03.581 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:03.581 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:03.581 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:03.594 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '58F602D7-2404-4234-8211-737FD1E360E5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112509), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:03.594 4076 3051113163944 MainThread DEBUG os_win._utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7285938262939453 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:03.594 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-1a0648ca-e6ca-448e-8a69-bac16b68eabf - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:03.594 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-1a0648ca-e6ca-448e-8a69-bac16b68eabf - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:03.628 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-1a0648ca-e6ca-448e-8a69-bac16b68eabf - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:03.685 4076 3051055214664 MainThread INFO nova.compute.manager [req-1a0648ca-e6ca-448e-8a69-bac16b68eabf - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:04.629 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '02A60C95-DF0C-4F9D-84F8-926E4DC2C29A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14818), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:04.755 4076 3051055214664 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:04.755 4076 3051055214664 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:04.755 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:04.771 4076 3051055215720 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.016s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:04.771 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '02A60C95-DF0C-4F9D-84F8-926E4DC2C29A', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111778), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:04.785 4076 3051113163944 MainThread DEBUG os_win._utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.919187545776367 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:04.785 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-b05f888d-0132-4e06-a7fd-070c122fedd0 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:04.785 4076 3051055214664 MainThread DEBUG nova.virt.hyperv.vmops [req-b05f888d-0132-4e06-a7fd-070c122fedd0 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:04.833 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-b05f888d-0132-4e06-a7fd-070c122fedd0 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:04.857 4076 3051055214664 MainThread INFO nova.compute.manager [req-b05f888d-0132-4e06-a7fd-070c122fedd0 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:05.801 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': 'BA339E05-0909-4CDE-91C3-7EE0CC606C81', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14493), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:05.829 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:05.829 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:39:05.907 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:39:05.907 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:39:05.924 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:05.924 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:05.953 4076 3051055217128 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:05.953 4076 3051055217128 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] VM Stopped (Lifecycle Event) 2021-09-10 17:39:05.953 4076 3051113163944 MainThread ERROR os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000092\' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000092\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)instance-000000929F90442F-5FBD-4634-BE07-470B7604DCEB%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'BA339E05-0909-4CDE-91C3-7EE0CC606C81', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112748), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 5), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\nNot enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)\n\n'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB)", 'ErrorSummaryDescription': "'instance-00000092' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000092.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:05.953 4076 3051113163944 MainThread ERROR nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Failed to change vm state of instance-00000092 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. 2021-09-10 17:39:05.953 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Got request to destroy instance 2021-09-10 17:39:05.967 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-202813b1-e909-4eca-809f-c6bb6225fd77 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:05.967 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-202813b1-e909-4eca-809f-c6bb6225fd77 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:06.017 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-202813b1-e909-4eca-809f-c6bb6225fd77 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:06.017 4076 3051055217128 MainThread INFO nova.compute.manager [req-202813b1-e909-4eca-809f-c6bb6225fd77 - - - - -] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:06.396 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:39:06.425 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:39:06.425 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:39:06.425 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:07.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:07.745 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:39:07.783 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:39:07.783 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:07.783 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:07.783 4076 3051113163944 MainThread DEBUG os_win.utils.compute.vmutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed vm state of instance-00000092 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:39:07.783 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Successfully changed state of VM instance-00000092 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:39:07.836 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Waiting for WMI job: {'InstanceID': '8AE3C453-4221-45BD-839C-C292F64C8F52', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14581), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:07.961 4076 3051113163944 MainThread DEBUG os_win.utils.jobutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] WMI job succeeded: {'InstanceID': '8AE3C453-4221-45BD-839C-C292F64C8F52', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=28488), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:39:07.961 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:07.961 4076 3051113163944 MainThread DEBUG os_win.utils.pathutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Removing directory: C:\openstack\instances\instance-00000092 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:39:07.961 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:07.961 4076 3051113163944 MainThread ERROR nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Traceback (most recent call last): 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(*args, **kwargs) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] admin_password, network_info, block_device_info) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(self, *args, **kwds) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] should_plug_vifs=False) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'req_state': req_state}) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] time.sleep(sleep_time) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return f(*args, **kwargs) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return self._wait_for_job(job_path) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] error_desc=err_desc) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During handling of the above exception, another exception occurred: 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Traceback (most recent call last): 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] yield resources 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] accel_info=accel_info) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(*args, **kwargs) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] admin_password, network_info, block_device_info) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(self, *args, **kwds) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] should_plug_vifs=False) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'req_state': req_state}) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] time.sleep(sleep_time) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return f(*args, **kwargs) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return self._wait_for_job(job_path) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] error_desc=err_desc) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:07.961 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:07.961 4076 3051113163944 MainThread INFO nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Terminating instance 2021-09-10 17:39:07.961 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:39:07.961 4076 3051113163944 MainThread INFO nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Got request to destroy instance 2021-09-10 17:39:07.989 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:39:07.989 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:07.989 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "instance-00000092" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:07.989 4076 3051113163944 MainThread INFO nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Took 0.03 seconds to destroy the instance on the hypervisor. 2021-09-10 17:39:07.989 4076 3051113163944 MainThread DEBUG nova.compute.claims [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:39:07.989 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:08.065 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:39:08.098 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.hostops [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:39:08.313 4076 3051113163944 MainThread DEBUG nova.compute.provider_tree [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.scheduler.client.report [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:08.328 4076 3051113163944 MainThread ERROR nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Traceback (most recent call last): 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(*args, **kwargs) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] admin_password, network_info, block_device_info) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(self, *args, **kwds) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] should_plug_vifs=False) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'req_state': req_state}) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] time.sleep(sleep_time) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return f(*args, **kwargs) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return self._wait_for_job(job_path) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] error_desc=err_desc) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During handling of the above exception, another exception occurred: 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Traceback (most recent call last): 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] accel_info=accel_info) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(*args, **kwargs) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] admin_password, network_info, block_device_info) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return function(self, *args, **kwds) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] should_plug_vifs=False) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'req_state': req_state}) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] time.sleep(sleep_time) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self.force_reraise() 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] raise self.value 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return f(*args, **kwargs) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] return self._wait_for_job(job_path) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] error_desc=err_desc) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. 2021-09-10 17:39:08.328 4076 3051113163944 MainThread TRACE nova.compute.manager [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.compute.utils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Build of instance 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000092' failed to start. Not enough memory in the system to start the virtual machine instance-00000092. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000092' failed to start. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Not enough memory in the system to start the virtual machine instance-00000092 with ram size 1024 megabytes. (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) 'instance-00000092' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 9F90442F-5FBD-4634-BE07-470B7604DCEB) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:39:08.328 4076 3051113163944 MainThread DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:39:08.801 4076 3051113163944 MainThread DEBUG nova.network.neutron [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:39:08.828 4076 3051113163944 MainThread INFO nova.compute.manager [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] Took 0.50 seconds to deallocate network for instance. 2021-09-10 17:39:08.985 4076 3051113163944 MainThread INFO nova.scheduler.client.report [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Deleted allocations for instance 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d 2021-09-10 17:39:09.033 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [req-5119bff7-94e6-4c68-b63d-ade8817dba00 580888399edc4d7c97b4a1bce73db000 351b14c1ba6b4a35aa1c79568c8e5c30 - default default] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.859s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:09.033 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 18.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:09.033 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 436b11e9-988c-46a7-a14a-9b5f6aa2bf2d] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:09.033 4076 3051055218008 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "436b11e9-988c-46a7-a14a-9b5f6aa2bf2d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:12.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:14.750 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:15.750 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:16.717 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:17.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:17.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:17.785 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:17.785 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:17.785 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:39:17.785 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:39:17.785 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:39:18.026 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2208MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:39:18.026 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:39:18.153 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:39:18.165 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:39:18.176 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:39:18.205 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:39:18.386 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:39:18.410 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:39:18.410 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:39:18.410 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:21.375 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:39:49.495 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:49.523 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:39:49.598 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:49.598 4076 3051055217128 MainThread DEBUG nova.virt.hardware [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:39:49.624 4076 3051055217128 MainThread INFO nova.compute.claims [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Claim successful on node n-h1-808474-1 2021-09-10 17:39:49.737 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:39:49.737 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:39:49.937 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:39:49.960 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:39:49.960 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:49.960 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:39:50.036 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:39:50.036 4076 3051055217304 GreenThread-135 DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:39:50.036 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:39:50.051 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:39:50.187 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:39:50.192 4076 3051055217304 GreenThread-135 DEBUG nova.policy [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ca5b31b9b47e4f338e12c70f0c03a712', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01d2edbd40ad476fb7f6b1f64070fee3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:39:50.192 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Spawning new instance 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG nova.block_device [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Creating directory: C:\openstack\instances\instance-00000093 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:39:50.204 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000093\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:39:50.234 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Resizing VHD C:\openstack\instances\instance-00000093\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:39:50.784 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Successfully created port: 5a1c384f-97bd-4de8-98be-555c8281d0bd _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:39:51.431 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Successfully updated port: 5a1c384f-97bd-4de8-98be-555c8281d0bd _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:39:51.441 4076 3051055217304 GreenThread-135 DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Acquired lock "refresh_cache-791f6785-0942-4130-b2e4-c5238398c1c8" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:39:51.441 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:39:51.474 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Received event network-changed-5a1c384f-97bd-4de8-98be-555c8281d0bd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:39:51.474 4076 3051113163416 MainThread DEBUG nova.compute.manager [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Refreshing instance network info cache due to event network-changed-5a1c384f-97bd-4de8-98be-555c8281d0bd. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:39:51.531 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:39:51.848 4076 3051055217304 GreenThread-135 DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Updating instance_info_cache with network_info: [{"id": "5a1c384f-97bd-4de8-98be-555c8281d0bd", "address": "fa:16:3e:90:6b:9c", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5a1c384f-97", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:39:51.874 4076 3051055217304 GreenThread-135 DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Releasing lock "refresh_cache-791f6785-0942-4130-b2e4-c5238398c1c8" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:39:51.874 4076 3051055217304 GreenThread-135 DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Instance network_info: |[{"id": "5a1c384f-97bd-4de8-98be-555c8281d0bd", "address": "fa:16:3e:90:6b:9c", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5a1c384f-97", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:39:51.874 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Preparing to wait for external event network-vif-plugged-5a1c384f-97bd-4de8-98be-555c8281d0bd prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:39:51.874 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:51.874 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:51.874 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Creating VM instance-00000093 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:39:51.874 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-791f6785-0942-4130-b2e4-c5238398c1c8" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:39:51.874 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Refreshing network info cache for port 5a1c384f-97bd-4de8-98be-555c8281d0bd _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:39:52.364 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Updated VIF entry in instance network info cache for port 5a1c384f-97bd-4de8-98be-555c8281d0bd. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:39:52.364 4076 3051113163416 MainThread DEBUG nova.network.neutron [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Updating instance_info_cache with network_info: [{"id": "5a1c384f-97bd-4de8-98be-555c8281d0bd", "address": "fa:16:3e:90:6b:9c", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5a1c384f-97", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:39:52.380 4076 3051113163416 MainThread DEBUG oslo_concurrency.lockutils [req-684c5b63-04c4-4b9f-b12a-782a63e3b975 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-791f6785-0942-4130-b2e4-c5238398c1c8" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:39:52.765 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:39:57.285 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-92cca3fe-c8ff-483a-90c2-e3163903e58f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Received event network-vif-plugged-5a1c384f-97bd-4de8-98be-555c8281d0bd external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:39:57.285 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-92cca3fe-c8ff-483a-90c2-e3163903e58f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:57.285 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-92cca3fe-c8ff-483a-90c2-e3163903e58f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:57.285 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-92cca3fe-c8ff-483a-90c2-e3163903e58f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Processing event network-vif-plugged-5a1c384f-97bd-4de8-98be-555c8281d0bd _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:39:57.312 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Using config drive for instance 2021-09-10 17:39:57.426 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Creating config drive at C:\openstack\instances\instance-00000093\configdrive.iso 2021-09-10 17:39:57.456 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000093\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4f4uof65 execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:39:57.769 4076 3051055217128 MainThread DEBUG oslo_concurrency.processutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000093\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmp4f4uof65" returned: 0 in 0.313s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:39:58.158 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Setting boot order for instance: instance-00000093: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:39:58.207 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:39:58.239 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'CB80E844-C9E5-44FD-A36F-2F631BA0613C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15170), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:58.383 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:58.383 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:58.383 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:58.383 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:39:58.391 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'CB80E844-C9E5-44FD-A36F-2F631BA0613C', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=120716), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 58), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 58), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:58.391 4076 3051055217128 MainThread DEBUG os_win._utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.1842937469482422 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:58.391 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-5f0bef69-3d86-4d9b-8302-ed41f7b25d99 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:58.391 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-5f0bef69-3d86-4d9b-8302-ed41f7b25d99 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:58.441 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-5f0bef69-3d86-4d9b-8302-ed41f7b25d99 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:58.468 4076 3051055218008 MainThread INFO nova.compute.manager [req-5f0bef69-3d86-4d9b-8302-ed41f7b25d99 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:39:59.415 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'D559800D-E292-4780-B181-7111678D5034', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15641), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:39:59.557 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:39:59.557 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:39:59.557 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:39:59.557 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:39:59.566 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D559800D-E292-4780-B181-7111678D5034', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113705), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 39, 59), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 39, 59), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:39:59.566 4076 3051055217128 MainThread DEBUG os_win._utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.35941743850708 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:39:59.582 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-7d5fb717-1c90-46e6-bf45-2de1aef8d937 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:39:59.582 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-7d5fb717-1c90-46e6-bf45-2de1aef8d937 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:39:59.629 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-7d5fb717-1c90-46e6-bf45-2de1aef8d937 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:39:59.643 4076 3051055218008 MainThread INFO nova.compute.manager [req-7d5fb717-1c90-46e6-bf45-2de1aef8d937 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:40:00.596 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': '5A9345EA-97F1-4076-BC17-CC25530B3A79', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19490), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:40:00.724 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:00.724 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:00.724 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:40:00.724 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:40:00.724 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-0b841d0c-f8bb-489f-96c8-911772a4a39c - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:40:00.749 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-0b841d0c-f8bb-489f-96c8-911772a4a39c - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:40:00.754 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '5A9345EA-97F1-4076-BC17-CC25530B3A79', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113198), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 40), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 40), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:40:00.754 4076 3051055217128 MainThread DEBUG os_win._utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.547520637512207 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:40:00.787 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-0b841d0c-f8bb-489f-96c8-911772a4a39c - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:40:00.814 4076 3051055218008 MainThread INFO nova.compute.manager [req-0b841d0c-f8bb-489f-96c8-911772a4a39c - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:40:01.785 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'D00C5733-B938-47C3-9A58-F37F2D5E6082', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15313), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:40:01.907 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:01.907 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:01.907 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:40:01.925 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:40:01.925 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'D00C5733-B938-47C3-9A58-F37F2D5E6082', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117697), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 40, 1), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 40, 1), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:40:01.925 4076 3051055217128 MainThread DEBUG os_win._utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.718796730041504 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:40:01.942 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-ae2537c1-dd3c-4a27-9c72-76d3950cb919 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:40:01.942 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-ae2537c1-dd3c-4a27-9c72-76d3950cb919 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:40:01.984 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-ae2537c1-dd3c-4a27-9c72-76d3950cb919 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:40:02.046 4076 3051055218008 MainThread INFO nova.compute.manager [req-ae2537c1-dd3c-4a27-9c72-76d3950cb919 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:40:02.952 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'A4E52649-D2A2-45DD-9299-AD2B21719DF1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15227), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:40:03.101 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:03.101 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:03.101 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:40:03.101 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:40:03.101 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A4E52649-D2A2-45DD-9299-AD2B21719DF1', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111904), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 40, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 40, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:40:03.101 4076 3051055217128 MainThread DEBUG os_win._utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.894544363021851 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:40:03.121 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-072dacba-e267-4d59-b16e-4c3822669174 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:40:03.121 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-072dacba-e267-4d59-b16e-4c3822669174 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:40:03.153 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-072dacba-e267-4d59-b16e-4c3822669174 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:40:03.186 4076 3051055218008 MainThread INFO nova.compute.manager [req-072dacba-e267-4d59-b16e-4c3822669174 - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:40:04.109 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'F10BE9A4-3A2A-4374-912B-00E01AB1AF83', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15600), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:40:04.259 4076 3051055218008 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:40:04.259 4076 3051055218008 MainThread INFO nova.compute.manager [-] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] VM Stopped (Lifecycle Event) 2021-09-10 17:40:04.259 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:04.259 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:04.259 4076 3051055217128 MainThread ERROR os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000093\' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000093\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)instance-00000093686CC669-5277-48FF-A8DE-C0814D9E20AD%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'F10BE9A4-3A2A-4374-912B-00E01AB1AF83', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112492), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 40, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 40, 4), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\nNot enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)\n\n'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD)", 'ErrorSummaryDescription': "'instance-00000093' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000093.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:40:04.259 4076 3051055217128 MainThread ERROR nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Failed to change vm state of instance-00000093 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. 2021-09-10 17:40:04.259 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Got request to destroy instance 2021-09-10 17:40:04.281 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-89ad2bb4-3731-491f-94bd-15f0a71c8f1b - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:40:04.281 4076 3051055218008 MainThread DEBUG nova.virt.hyperv.vmops [req-89ad2bb4-3731-491f-94bd-15f0a71c8f1b - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:40:04.327 4076 3051055218008 MainThread DEBUG nova.compute.manager [req-89ad2bb4-3731-491f-94bd-15f0a71c8f1b - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:40:04.350 4076 3051055218008 MainThread INFO nova.compute.manager [req-89ad2bb4-3731-491f-94bd-15f0a71c8f1b - - - - -] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:40:06.066 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:40:06.066 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:06.066 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:06.066 4076 3051055217128 MainThread DEBUG os_win.utils.compute.vmutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Successfully changed vm state of instance-00000093 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:40:06.066 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Successfully changed state of VM instance-00000093 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:40:06.140 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'BD1BCD0E-4E02-405B-9336-0C9A9ED7DE11', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14790), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:40:06.265 4076 3051055217128 MainThread DEBUG os_win.utils.jobutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job succeeded: {'InstanceID': 'BD1BCD0E-4E02-405B-9336-0C9A9ED7DE11', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29396), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:40:06.265 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:06.265 4076 3051055217128 MainThread DEBUG os_win.utils.pathutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Removing directory: C:\openstack\instances\instance-00000093 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:40:06.265 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:06.265 4076 3051055217128 MainThread ERROR nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Traceback (most recent call last): 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(*args, **kwargs) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] admin_password, network_info, block_device_info) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(self, *args, **kwds) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] should_plug_vifs=False) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'req_state': req_state}) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] time.sleep(sleep_time) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return f(*args, **kwargs) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return self._wait_for_job(job_path) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] error_desc=err_desc) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During handling of the above exception, another exception occurred: 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Traceback (most recent call last): 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] yield resources 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] accel_info=accel_info) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(*args, **kwargs) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] admin_password, network_info, block_device_info) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(self, *args, **kwds) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] should_plug_vifs=False) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'req_state': req_state}) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] time.sleep(sleep_time) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return f(*args, **kwargs) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return self._wait_for_job(job_path) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] error_desc=err_desc) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.265 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.265 4076 3051055217128 MainThread INFO nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Terminating instance 2021-09-10 17:40:06.265 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:40:06.265 4076 3051055217128 MainThread INFO nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Got request to destroy instance 2021-09-10 17:40:06.280 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.vmops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:40:06.280 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:06.280 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000093" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:06.280 4076 3051055217128 MainThread INFO nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 17:40:06.296 4076 3051055217128 MainThread DEBUG nova.compute.claims [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:40:06.296 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:06.375 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:40:06.395 4076 3051055217128 MainThread DEBUG nova.virt.hyperv.hostops [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:40:06.603 4076 3051055217128 MainThread DEBUG nova.compute.provider_tree [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.scheduler.client.report [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:06.618 4076 3051055217128 MainThread ERROR nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Traceback (most recent call last): 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(*args, **kwargs) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] admin_password, network_info, block_device_info) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(self, *args, **kwds) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] should_plug_vifs=False) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'req_state': req_state}) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] time.sleep(sleep_time) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return f(*args, **kwargs) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return self._wait_for_job(job_path) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] error_desc=err_desc) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] During handling of the above exception, another exception occurred: 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Traceback (most recent call last): 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] accel_info=accel_info) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(*args, **kwargs) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] admin_password, network_info, block_device_info) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return function(self, *args, **kwds) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] should_plug_vifs=False) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'req_state': req_state}) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] time.sleep(sleep_time) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self.force_reraise() 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] raise self.value 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return f(*args, **kwargs) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] return self._wait_for_job(job_path) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] error_desc=err_desc) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. 2021-09-10 17:40:06.618 4076 3051055217128 MainThread TRACE nova.compute.manager [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.compute.utils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Build of instance 791f6785-0942-4130-b2e4-c5238398c1c8 was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000093' failed to start. Not enough memory in the system to start the virtual machine instance-00000093. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000093' failed to start. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Not enough memory in the system to start the virtual machine instance-00000093 with ram size 1024 megabytes. (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) 'instance-00000093' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 686CC669-5277-48FF-A8DE-C0814D9E20AD) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:40:06.618 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:40:07.406 4076 3051055217128 MainThread DEBUG nova.network.neutron [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:40:07.406 4076 3051055217128 MainThread INFO nova.compute.manager [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: 791f6785-0942-4130-b2e4-c5238398c1c8] Took 0.78 seconds to deallocate network for instance. 2021-09-10 17:40:07.593 4076 3051055217128 MainThread INFO nova.scheduler.client.report [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Deleted allocations for instance 791f6785-0942-4130-b2e4-c5238398c1c8 2021-09-10 17:40:07.629 4076 3051055217128 MainThread DEBUG oslo_concurrency.lockutils [req-cb51134a-d223-4c13-b746-b80b9836905a ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "791f6785-0942-4130-b2e4-c5238398c1c8" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.125s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:07.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:07.754 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:40:07.973 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:40:07.973 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:40:08.609 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:40:08.624 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:40:08.624 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:40:08.624 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:08.755 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:08.755 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:40:14.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:15.749 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:16.711 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:16.967 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:17.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:18.769 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:18.781 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:18.781 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:18.781 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:40:18.781 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:40:18.801 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:40:19.016 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2212MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:40:19.016 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:19.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:40:19.159 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:40:19.191 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:40:19.202 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:40:19.394 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:40:19.408 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:40:19.408 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:40:19.408 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.390s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:20.380 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:40:46.770 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:46.785 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:40:46.880 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:46.880 4076 3051055217304 MainThread DEBUG nova.virt.hardware [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:40:46.903 4076 3051055217304 MainThread INFO nova.compute.claims [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Claim successful on node n-h1-808474-1 2021-09-10 17:40:46.989 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:40:47.020 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:40:47.215 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:40:47.236 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:40:47.236 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:47.236 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:40:47.297 4076 3051055217304 MainThread DEBUG nova.compute.utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:40:47.297 4076 3051055218008 GreenThread-136 DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:40:47.297 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:40:47.311 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:40:47.364 4076 3051055217304 MainThread INFO nova.virt.block_device [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Booting with volume 26a22bd2-cb94-4736-8838-312896b8255c at /dev/sda 2021-09-10 17:40:47.427 4076 3051055218008 GreenThread-136 DEBUG nova.policy [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ca5b31b9b47e4f338e12c70f0c03a712', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '01d2edbd40ad476fb7f6b1f64070fee3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:40:47.427 4076 3051055217304 MainThread DEBUG os_brick.utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] ==> get_connector_properties: call "{'root_helper': None, 'my_ip': '10.222.0.32', 'multipath': False, 'enforce_multipath': True, 'host': 'n-h1-808474-1', 'execute': None}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:40:47.427 4076 3051055217304 MainThread DEBUG os_brick.utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] <== get_connector_properties: return (0ms) {'platform': 'AMD64', 'os_type': 'win32', 'ip': '10.222.0.32', 'host': 'n-h1-808474-1', 'multipath': False, 'initiator': 'iqn.1991-05.com.microsoft:n-h1-808474-1.cbci-808474-1.local'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:40:47.427 4076 3051055217304 MainThread DEBUG nova.virt.block_device [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Updating existing volume attachment record: 09792c9a-7fb6-4733-ae27-2e064cf94b09 _volume_attach c:\openstack\build\nova\nova\virt\block_device.py:556 2021-09-10 17:40:48.036 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Successfully created port: 9acbf57e-aadc-4f50-b913-2b11bf86846f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:40:48.687 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Successfully updated port: 9acbf57e-aadc-4f50-b913-2b11bf86846f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:40:48.710 4076 3051055218008 GreenThread-136 DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Acquired lock "refresh_cache-e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:40:48.710 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:40:48.738 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Received event network-changed-9acbf57e-aadc-4f50-b913-2b11bf86846f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:40:48.754 4076 3051113162888 MainThread DEBUG nova.compute.manager [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Refreshing instance network info cache due to event network-changed-9acbf57e-aadc-4f50-b913-2b11bf86846f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:40:48.785 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:40:49.437 4076 3051055218008 GreenThread-136 DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Updating instance_info_cache with network_info: [{"id": "9acbf57e-aadc-4f50-b913-2b11bf86846f", "address": "fa:16:3e:c0:15:c1", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9acbf57e-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:40:49.458 4076 3051055218008 GreenThread-136 DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Releasing lock "refresh_cache-e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:40:49.458 4076 3051055218008 GreenThread-136 DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Instance network_info: |[{"id": "9acbf57e-aadc-4f50-b913-2b11bf86846f", "address": "fa:16:3e:c0:15:c1", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9acbf57e-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:40:49.458 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:40:49.458 4076 3051113162888 MainThread DEBUG nova.network.neutron [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Refreshing network info cache for port 9acbf57e-aadc-4f50-b913-2b11bf86846f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:40:49.958 4076 3051113162888 MainThread DEBUG nova.network.neutron [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Updated VIF entry in instance network info cache for port 9acbf57e-aadc-4f50-b913-2b11bf86846f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:40:49.958 4076 3051113162888 MainThread DEBUG nova.network.neutron [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Updating instance_info_cache with network_info: [{"id": "9acbf57e-aadc-4f50-b913-2b11bf86846f", "address": "fa:16:3e:c0:15:c1", "network": {"id": "88182bf3-791a-48fd-ad45-bb3de5fa5f37", "bridge": null, "label": "tempest-TestShelveInstance-977255127-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "01d2edbd40ad476fb7f6b1f64070fee3", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9acbf57e-aa", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:40:49.989 4076 3051113162888 MainThread DEBUG oslo_concurrency.lockutils [req-bd50956d-06f0-4dec-aacf-796dc9fee6b4 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:40:51.482 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:40:51.495 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Spawning new instance 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG nova.block_device [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] block_device_list ['sda'] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Preparing to wait for external event network-vif-plugged-9acbf57e-aadc-4f50-b913-2b11bf86846f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:40:51.514 4076 3051055217304 MainThread DEBUG os_win.utils.compute.vmutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Creating VM instance-00000094 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:40:51.942 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.volumeops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Attaching volume: {'driver_volume_type': 'iscsi', 'data': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c', 'target_lun': 0, 'volume_id': '26a22bd2-cb94-4736-8838-312896b8255c', 'auth_method': 'CHAP', 'auth_username': 'Dy2Qf7XCEFUbu6eKJt2D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'status': 'reserved', 'instance': 'e373fdf1-1fae-4b31-9dd1-30a4529ed1ca', 'attached_at': '', 'detached_at': '', 'volume_id': '26a22bd2-cb94-4736-8838-312896b8255c', 'serial': '26a22bd2-cb94-4736-8838-312896b8255c'} to instance-00000094 _attach_volume c:\openstack\build\nova\nova\virt\hyperv\volumeops.py:108 2021-09-10 17:40:51.942 4076 3051055217304 MainThread DEBUG os_brick.initiator.windows.iscsi [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c', 'target_lun': 0, 'volume_id': '26a22bd2-cb94-4736-8838-312896b8255c', 'auth_method': 'CHAP', 'auth_username': 'Dy2Qf7XCEFUbu6eKJt2D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:40:51.942 4076 3051055217304 MainThread INFO os_brick.initiator.windows.iscsi [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Attempting to establish an iSCSI session to target iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c on portal 10.222.0.55:3260 accessing LUN 0 using initiator None. 2021-09-10 17:40:51.942 4076 3051055217304 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Logging in iSCSI target iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c login_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:401 2021-09-10 17:40:56.967 4076 3051055217304 MainThread DEBUG os_win.utils.storage.diskutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:40:57.102 4076 3051055217304 MainThread DEBUG os_win.utils.storage.diskutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:40:57.155 4076 3051055217304 MainThread DEBUG os_brick.initiator.windows.iscsi [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] <== connect_volume: return (5212ms) {'type': 'block', 'path': '\\\\.\\PhysicalDrive1', 'number': 1, 'scsi_wwn': '36001405CE9748D0BB6141A698AFB7893'} trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:40:57.582 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:41:01.582 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-fd3e46d0-2513-4d63-89e1-dc764fe48cde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Received event network-vif-plugged-9acbf57e-aadc-4f50-b913-2b11bf86846f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:41:01.582 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-fd3e46d0-2513-4d63-89e1-dc764fe48cde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:01.582 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [req-fd3e46d0-2513-4d63-89e1-dc764fe48cde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:01.582 4076 3051055214664 MainThread DEBUG nova.compute.manager [req-fd3e46d0-2513-4d63-89e1-dc764fe48cde 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Processing event network-vif-plugged-9acbf57e-aadc-4f50-b913-2b11bf86846f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:41:01.598 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Using config drive for instance 2021-09-10 17:41:01.716 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Creating config drive at C:\openstack\instances\instance-00000094\configdrive.iso 2021-09-10 17:41:01.758 4076 3051055217304 MainThread DEBUG oslo_concurrency.processutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000094\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplap81j_w execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:41:02.051 4076 3051055217304 MainThread DEBUG oslo_concurrency.processutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000094\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmplap81j_w" returned: 0 in 0.281s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:41:02.457 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Setting boot order for instance: instance-00000094: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:41:02.504 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:41:02.539 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'E45AEB69-7669-4A16-8138-5874B3DE4517', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=18792), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:02.673 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:02.673 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:02.673 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:02.673 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:02.698 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-ce018279-ed28-40b9-b6d9-ffee4f22bd89 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:02.698 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-ce018279-ed28-40b9-b6d9-ffee4f22bd89 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:02.702 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'E45AEB69-7669-4A16-8138-5874B3DE4517', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=114380), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 2), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 2), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:02.702 4076 3051055217304 MainThread DEBUG os_win._utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.19781208038330078 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:41:02.736 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-ce018279-ed28-40b9-b6d9-ffee4f22bd89 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:02.769 4076 3051055215720 MainThread INFO nova.compute.manager [req-ce018279-ed28-40b9-b6d9-ffee4f22bd89 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:03.722 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': '719D550A-60BF-4C7D-9B11-93187ED28279', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15618), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:03.862 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:03.862 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:03.862 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:03.862 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:03.877 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '719D550A-60BF-4C7D-9B11-93187ED28279', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111991), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 3), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 3), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:03.877 4076 3051055217304 MainThread DEBUG os_win._utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.3728358745574951 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:41:03.877 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-01c6d1c0-5439-4a5f-a1d9-31fea81be726 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:03.877 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-01c6d1c0-5439-4a5f-a1d9-31fea81be726 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:03.920 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-01c6d1c0-5439-4a5f-a1d9-31fea81be726 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:03.951 4076 3051055215720 MainThread INFO nova.compute.manager [req-01c6d1c0-5439-4a5f-a1d9-31fea81be726 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:04.915 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': '618CC99A-456B-4286-ADAB-CF6CA1CB9AB6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15515), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:05.053 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:05.053 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:05.053 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:05.053 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:05.060 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-8c152231-c533-414e-8dbe-ab145f8e93bd - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:05.060 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-8c152231-c533-414e-8dbe-ab145f8e93bd - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:05.060 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '618CC99A-456B-4286-ADAB-CF6CA1CB9AB6', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112707), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 4), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 5), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:05.060 4076 3051055217304 MainThread DEBUG os_win._utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5562260150909424 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:41:05.117 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-8c152231-c533-414e-8dbe-ab145f8e93bd - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:05.143 4076 3051055215720 MainThread INFO nova.compute.manager [req-8c152231-c533-414e-8dbe-ab145f8e93bd - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:06.091 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': '1885E582-59E1-4451-940A-3E4DE9C7C3F5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14955), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:06.223 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:06.223 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:06.236 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:06.236 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:06.236 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '1885E582-59E1-4451-940A-3E4DE9C7C3F5', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=117002), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 6), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 6), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:06.236 4076 3051055217304 MainThread DEBUG os_win._utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.7314462661743164 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:41:06.236 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-3044aa0c-ce9a-4b7a-84a7-3638012d3b54 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:06.254 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-3044aa0c-ce9a-4b7a-84a7-3638012d3b54 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:06.301 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-3044aa0c-ce9a-4b7a-84a7-3638012d3b54 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:06.316 4076 3051055215720 MainThread INFO nova.compute.manager [req-3044aa0c-ce9a-4b7a-84a7-3638012d3b54 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:07.269 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': '8B16A3BC-0EE2-4666-AA4A-F8A5F1D06E5D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14885), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:07.413 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:07.413 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:07.413 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:07.413 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:07.413 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8B16A3BC-0EE2-4666-AA4A-F8A5F1D06E5D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112335), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 7), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 7), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:07.420 4076 3051055217304 MainThread DEBUG os_win._utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.915555953979492 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:41:07.420 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-76464f25-2021-48c8-918f-bf315ce1c9e4 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:07.420 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-76464f25-2021-48c8-918f-bf315ce1c9e4 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:07.478 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-76464f25-2021-48c8-918f-bf315ce1c9e4 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:07.491 4076 3051055215720 MainThread INFO nova.compute.manager [req-76464f25-2021-48c8-918f-bf315ce1c9e4 - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:07.770 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:08.443 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'DD05ADCB-94E7-42E6-AE60-CE90CFD2994B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15213), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:08.583 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:08.583 4076 3051055214664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:08.593 4076 3051055215720 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:41:08.593 4076 3051055215720 MainThread INFO nova.compute.manager [-] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] VM Stopped (Lifecycle Event) 2021-09-10 17:41:08.593 4076 3051055217304 MainThread ERROR os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000094\' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA612030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA610243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000094\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)instance-000000945F66158D-CA20-4FF0-B1ED-3CD778674BA6%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'DD05ADCB-94E7-42E6-AE60-CE90CFD2994B', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112929), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 41, 8), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 41, 8), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\nNot enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)\n\n'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6)", 'ErrorSummaryDescription': "'instance-00000094' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000094.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:41:08.593 4076 3051055217304 MainThread ERROR nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Failed to change vm state of instance-00000094 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. 2021-09-10 17:41:08.593 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Got request to destroy instance 2021-09-10 17:41:08.593 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-0587570d-75db-417e-a944-4b933a71084c - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:41:08.593 4076 3051055215720 MainThread DEBUG nova.virt.hyperv.vmops [req-0587570d-75db-417e-a944-4b933a71084c - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:41:08.662 4076 3051055215720 MainThread DEBUG nova.compute.manager [req-0587570d-75db-417e-a944-4b933a71084c - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:41:08.682 4076 3051055215720 MainThread INFO nova.compute.manager [req-0587570d-75db-417e-a944-4b933a71084c - - - - -] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:41:09.748 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:09.748 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:41:09.823 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:41:09.823 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:41:10.314 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updating instance_info_cache with network_info: [{"id": "9ce6624e-4307-41bc-9d39-c5c0bd6572b9", "address": "fa:16:3e:1a:ba:e3", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap9ce6624e-43", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:41:10.314 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-17abaf80-43c7-4c09-b4a2-c12636cd0cc3" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:41:10.314 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:41:10.407 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:41:10.407 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:10.407 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:10.433 4076 3051055217304 MainThread DEBUG os_win.utils.compute.vmutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Successfully changed vm state of instance-00000094 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:41:10.433 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Successfully changed state of VM instance-00000094 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:41:10.491 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Waiting for WMI job: {'InstanceID': 'F39DBD0C-AAB0-4D82-A5F9-75D392BFB277', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14245), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:41:10.615 4076 3051055217304 MainThread DEBUG os_win.utils.jobutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] WMI job succeeded: {'InstanceID': 'F39DBD0C-AAB0-4D82-A5F9-75D392BFB277', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=30776), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:41:10.615 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:10.615 4076 3051055217304 MainThread DEBUG os_win.utils.pathutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Removing directory: C:\openstack\instances\instance-00000094 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:41:10.615 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:10.615 4076 3051055217304 MainThread ERROR nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Traceback (most recent call last): 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(*args, **kwargs) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] admin_password, network_info, block_device_info) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(self, *args, **kwds) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] should_plug_vifs=False) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'req_state': req_state}) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] time.sleep(sleep_time) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return f(*args, **kwargs) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return self._wait_for_job(job_path) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] error_desc=err_desc) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During handling of the above exception, another exception occurred: 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Traceback (most recent call last): 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] yield resources 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] accel_info=accel_info) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(*args, **kwargs) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] admin_password, network_info, block_device_info) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(self, *args, **kwds) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] should_plug_vifs=False) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'req_state': req_state}) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] time.sleep(sleep_time) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return f(*args, **kwargs) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return self._wait_for_job(job_path) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] error_desc=err_desc) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:10.615 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:10.629 4076 3051055217304 MainThread INFO nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Terminating instance 2021-09-10 17:41:10.629 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:41:10.629 4076 3051055217304 MainThread INFO nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Got request to destroy instance 2021-09-10 17:41:10.644 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:41:10.644 4076 3051055217304 MainThread DEBUG os_brick.initiator.windows.iscsi [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] ==> disconnect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '10.222.0.55:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c', 'target_lun': 0, 'volume_id': '26a22bd2-cb94-4736-8838-312896b8255c', 'auth_method': 'CHAP', 'auth_username': 'Dy2Qf7XCEFUbu6eKJt2D', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}, 'device_info': None, 'force': False, 'ignore_errors': False}" trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:163 2021-09-10 17:41:10.644 4076 3051055217304 MainThread DEBUG os_win.utils.storage.diskutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:199 2021-09-10 17:41:10.738 4076 3051055217304 MainThread DEBUG os_win.utils.storage.diskutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Finished rescanning disks. _rescan_disks c:\openstack\build\os-win\os_win\utils\storage\diskutils.py:210 2021-09-10 17:41:10.738 4076 3051055217304 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Logging out iSCSI target iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c logout_storage_target c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:480 2021-09-10 17:41:10.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:10.758 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:41:10.823 4076 3051055217304 MainThread DEBUG os_win.utils.storage.initiator.iscsi_utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Removing iSCSI target persistent login: iqn.2010-10.org.openstack:volume-26a22bd2-cb94-4736-8838-312896b8255c _remove_target_persistent_logins c:\openstack\build\os-win\os_win\utils\storage\initiator\iscsi_utils.py:495 2021-09-10 17:41:10.823 4076 3051055217304 MainThread DEBUG os_brick.initiator.windows.iscsi [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] <== disconnect_volume: return (178ms) None trace_logging_wrapper c:\python37\lib\site-packages\os_brick\utils.py:187 2021-09-10 17:41:10.823 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:10.823 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "instance-00000094" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:10.823 4076 3051055217304 MainThread INFO nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Took 0.20 seconds to destroy the instance on the hypervisor. 2021-09-10 17:41:14.034 4076 3051055217304 MainThread INFO nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Took 3.20 seconds to detach 1 volumes for instance. 2021-09-10 17:41:14.034 4076 3051055217304 MainThread DEBUG nova.compute.claims [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:41:14.034 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:14.138 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:41:14.158 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.hostops [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:41:14.363 4076 3051055217304 MainThread DEBUG nova.compute.provider_tree [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.scheduler.client.report [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:14.377 4076 3051055217304 MainThread ERROR nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Traceback (most recent call last): 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(*args, **kwargs) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] admin_password, network_info, block_device_info) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(self, *args, **kwds) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] should_plug_vifs=False) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'req_state': req_state}) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] time.sleep(sleep_time) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return f(*args, **kwargs) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return self._wait_for_job(job_path) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] error_desc=err_desc) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] During handling of the above exception, another exception occurred: 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Traceback (most recent call last): 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] accel_info=accel_info) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(*args, **kwargs) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] admin_password, network_info, block_device_info) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return function(self, *args, **kwds) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] should_plug_vifs=False) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'req_state': req_state}) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] time.sleep(sleep_time) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self.force_reraise() 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] raise self.value 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return f(*args, **kwargs) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] return self._wait_for_job(job_path) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] error_desc=err_desc) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. 2021-09-10 17:41:14.377 4076 3051055217304 MainThread TRACE nova.compute.manager [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.compute.utils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Build of instance e373fdf1-1fae-4b31-9dd1-30a4529ed1ca was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000094' failed to start. Not enough memory in the system to start the virtual machine instance-00000094. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000094' failed to start. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Not enough memory in the system to start the virtual machine instance-00000094 with ram size 1024 megabytes. (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) 'instance-00000094' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 5F66158D-CA20-4FF0-B1ED-3CD778674BA6) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:41:14.377 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:41:14.860 4076 3051055217304 MainThread DEBUG nova.network.neutron [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:41:14.878 4076 3051055217304 MainThread INFO nova.compute.manager [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] [instance: e373fdf1-1fae-4b31-9dd1-30a4529ed1ca] Took 0.50 seconds to deallocate network for instance. 2021-09-10 17:41:15.030 4076 3051055217304 MainThread INFO nova.scheduler.client.report [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Deleted allocations for instance e373fdf1-1fae-4b31-9dd1-30a4529ed1ca 2021-09-10 17:41:15.063 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-74865e52-9050-4af9-a3f0-668bdc0ebe67 ca5b31b9b47e4f338e12c70f0c03a712 01d2edbd40ad476fb7f6b1f64070fee3 - default default] Lock "e373fdf1-1fae-4b31-9dd1-30a4529ed1ca" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 28.297s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:16.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:16.749 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:17.721 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:41:19.754 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:41:19.779 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:41:20.006 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2207MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:41:20.006 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.134 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:41:20.139 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:41:20.154 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:41:20.172 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:41:20.358 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:41:20.388 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:41:20.388 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:41:20.388 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:41:21.403 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:41:23.726 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:05.571 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:05.596 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Starting instance... _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2197 2021-09-10 17:42:05.691 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:05.691 4076 3051113163240 MainThread DEBUG nova.virt.hardware [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Require both a host and instance NUMA topology to fit instance on host. numa_fit_instance_to_host c:\openstack\build\nova\nova\virt\hardware.py:2177 2021-09-10 17:42:05.691 4076 3051113163240 MainThread INFO nova.compute.claims [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Claim successful on node n-h1-808474-1 2021-09-10 17:42:05.800 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:42:05.829 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:42:06.034 4076 3051113163240 MainThread DEBUG nova.compute.provider_tree [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:42:06.034 4076 3051113163240 MainThread DEBUG nova.scheduler.client.report [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:42:06.053 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:06.053 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Start building networks asynchronously for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2576 2021-09-10 17:42:06.113 4076 3051113163240 MainThread DEBUG nova.compute.utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Using /dev/sd instead of None get_next_device_name c:\openstack\build\nova\nova\compute\utils.py:239 2021-09-10 17:42:06.113 4076 3051055217304 GreenThread-137 DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Allocating IP information in the background. _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1756 2021-09-10 17:42:06.113 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] allocate_for_instance() allocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1116 2021-09-10 17:42:06.133 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Start building block device mappings for instance. _build_resources c:\openstack\build\nova\nova\compute\manager.py:2611 2021-09-10 17:42:06.237 4076 3051055217304 GreenThread-137 DEBUG nova.policy [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '10226e0397b7435499d0680801742547', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3efd33abd6ee4c68bcbc61a4464c4134', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} authorize c:\openstack\build\nova\nova\policy.py:202 2021-09-10 17:42:06.257 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Start spawning the instance on the hypervisor. _build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2396 2021-09-10 17:42:06.278 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Spawning new instance 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG nova.block_device [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] block_device_list [] volume_in_mapping c:\openstack\build\nova\nova\block_device.py:593 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" acquired by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2db44274-f9cc-4e8a-a37d-2b6502ddf47d-cache.lock" released by "nova.virt.hyperv.imagecache.ImageCache.get_cached_image..fetch_image_if_not_existing" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG os_win.utils.pathutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Creating directory: C:\openstack\instances\instance-00000095 check_create_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:146 2021-09-10 17:42:06.281 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Creating differencing VHD. Parent: C:\openstack\instances\_base\2db44274-f9cc-4e8a-a37d-2b6502ddf47d.vhdx, Target: C:\openstack\instances\instance-00000095\root.vhdx _create_root_vhd c:\openstack\build\nova\nova\virt\hyperv\vmops.py:158 2021-09-10 17:42:06.312 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Resizing VHD C:\openstack\instances\instance-00000095\root.vhdx to new size 1065353216 _is_resize_needed c:\openstack\build\nova\nova\virt\hyperv\vmops.py:201 2021-09-10 17:42:07.207 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Successfully created port: 84882b91-a247-4d1b-a46d-18a06f074c3f _create_port_minimal c:\openstack\build\nova\nova\network\neutron.py:533 2021-09-10 17:42:07.746 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:07.861 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Successfully updated port: 84882b91-a247-4d1b-a46d-18a06f074c3f _update_port c:\openstack\build\nova\nova\network\neutron.py:571 2021-09-10 17:42:07.904 4076 3051055217304 GreenThread-137 DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Acquired lock "refresh_cache-2423d019-a7f7-433a-9baf-c071866eaccd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:42:07.904 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Building network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1877 2021-09-10 17:42:07.939 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Received event network-changed-84882b91-a247-4d1b-a46d-18a06f074c3f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:42:07.939 4076 3051113163064 MainThread DEBUG nova.compute.manager [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Refreshing instance network info cache due to event network-changed-84882b91-a247-4d1b-a46d-18a06f074c3f. external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10511 2021-09-10 17:42:07.965 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Instance cache missing network info. _get_preexisting_port_ids c:\openstack\build\nova\nova\network\neutron.py:3153 2021-09-10 17:42:08.299 4076 3051055217304 GreenThread-137 DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Updating instance_info_cache with network_info: [{"id": "84882b91-a247-4d1b-a46d-18a06f074c3f", "address": "fa:16:3e:03:9b:1d", "network": {"id": "9c9ec601-183d-4fbc-9575-40c57d66c504", "bridge": null, "label": "tempest-TestSnapshotPattern-1357695525-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "3efd33abd6ee4c68bcbc61a4464c4134", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap84882b91-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:42:08.340 4076 3051055217304 GreenThread-137 DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Releasing lock "refresh_cache-2423d019-a7f7-433a-9baf-c071866eaccd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:42:08.340 4076 3051055217304 GreenThread-137 DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Instance network_info: |[{"id": "84882b91-a247-4d1b-a46d-18a06f074c3f", "address": "fa:16:3e:03:9b:1d", "network": {"id": "9c9ec601-183d-4fbc-9575-40c57d66c504", "bridge": null, "label": "tempest-TestSnapshotPattern-1357695525-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "3efd33abd6ee4c68bcbc61a4464c4134", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap84882b91-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| _allocate_network_async c:\openstack\build\nova\nova\compute\manager.py:1771 2021-09-10 17:42:08.340 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Acquired lock "refresh_cache-2423d019-a7f7-433a-9baf-c071866eaccd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:42:08.340 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Refreshing network info cache for port 84882b91-a247-4d1b-a46d-18a06f074c3f _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1874 2021-09-10 17:42:08.347 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Preparing to wait for external event network-vif-plugged-84882b91-a247-4d1b-a46d-18a06f074c3f prepare_for_instance_event c:\openstack\build\nova\nova\compute\manager.py:281 2021-09-10 17:42:08.347 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:08.347 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd-events" released by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.._create_or_get_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:08.347 4076 3051113163240 MainThread DEBUG os_win.utils.compute.vmutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Creating VM instance-00000095 create_vm c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:353 2021-09-10 17:42:08.858 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Updated VIF entry in instance network info cache for port 84882b91-a247-4d1b-a46d-18a06f074c3f. _build_network_info_model c:\openstack\build\nova\nova\network\neutron.py:3288 2021-09-10 17:42:08.858 4076 3051113163064 MainThread DEBUG nova.network.neutron [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Updating instance_info_cache with network_info: [{"id": "84882b91-a247-4d1b-a46d-18a06f074c3f", "address": "fa:16:3e:03:9b:1d", "network": {"id": "9c9ec601-183d-4fbc-9575-40c57d66c504", "bridge": null, "label": "tempest-TestSnapshotPattern-1357695525-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "3efd33abd6ee4c68bcbc61a4464c4134", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap84882b91-a2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:42:08.889 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [req-160aa02d-cf5e-43c4-9f15-bcdb16a4c743 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Releasing lock "refresh_cache-2423d019-a7f7-433a-9baf-c071866eaccd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:42:09.237 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Creating nic for instance create_instance c:\openstack\build\nova\nova\virt\hyperv\vmops.py:421 2021-09-10 17:42:10.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:10.745 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:42:10.824 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:42:10.824 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:42:11.346 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updating instance_info_cache with network_info: [{"id": "c35b0100-fb8c-4ff3-9f5e-1faa360d72c4", "address": "fa:16:3e:61:5f:6b", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.103", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc35b0100-fb", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:42:11.361 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-ca35d63c-0cc3-4334-9b0d-fc30961067d5" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:42:11.361 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:42:11.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:11.753 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:42:13.989 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-d370d929-72a6-4693-b9f5-9e07cc5f471f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Received event network-vif-plugged-84882b91-a247-4d1b-a46d-18a06f074c3f external_instance_event c:\openstack\build\nova\nova\compute\manager.py:10507 2021-09-10 17:42:13.989 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-d370d929-72a6-4693-b9f5-9e07cc5f471f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:13.989 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [req-d370d929-72a6-4693-b9f5-9e07cc5f471f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd-events" released by "nova.compute.manager.InstanceEvents.pop_instance_event.._pop_event" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:13.989 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-d370d929-72a6-4693-b9f5-9e07cc5f471f 066bb0a533264941b909702b76848d2e 5fb6e4652c6846e2b642a48e1ed4ccae - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Processing event network-vif-plugged-84882b91-a247-4d1b-a46d-18a06f074c3f _process_instance_event c:\openstack\build\nova\nova\compute\manager.py:10266 2021-09-10 17:42:14.020 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Using config drive for instance 2021-09-10 17:42:14.133 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Creating config drive at C:\openstack\instances\instance-00000095\configdrive.iso 2021-09-10 17:42:14.165 4076 3051113163240 MainThread DEBUG oslo_concurrency.processutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Running cmd (subprocess): c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000095\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpcp5e6v1c execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:384 2021-09-10 17:42:14.482 4076 3051113163240 MainThread DEBUG oslo_concurrency.processutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] CMD "c:\openstack\bin\mkisofs.exe -o C:\openstack\instances\instance-00000095\configdrive.iso -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 23.1.0 -quiet -J -r -V config-2 C:\Users\ADMINI~1\AppData\Local\Temp\tmpcp5e6v1c" returned: 0 in 0.312s execute c:\python37\lib\site-packages\oslo_concurrency\processutils.py:423 2021-09-10 17:42:14.861 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Setting boot order for instance: instance-00000095: [2, 1, 3, 0] set_boot_order c:\openstack\build\nova\nova\virt\hyperv\vmops.py:252 2021-09-10 17:42:14.910 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Power on instance power_on c:\openstack\build\nova\nova\virt\hyperv\vmops.py:860 2021-09-10 17:42:14.943 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '74DE4EF6-F328-4D75-BDCD-32A66F6F61DE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=19935), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:15.060 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:15.060 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:15.060 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:15.060 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:15.090 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '74DE4EF6-F328-4D75-BDCD-32A66F6F61DE', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112628), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 14), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 15), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:15.090 4076 3051113163240 MainThread DEBUG os_win._utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 5. Time left: undefined. Time elapsed: 0.18015074729919434 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:42:15.102 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-f0e916e3-f0ee-4f05-b02c-b09e5c31447d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:15.102 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-f0e916e3-f0ee-4f05-b02c-b09e5c31447d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:15.152 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-f0e916e3-f0ee-4f05-b02c-b09e5c31447d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:15.168 4076 3051055217304 MainThread INFO nova.compute.manager [req-f0e916e3-f0ee-4f05-b02c-b09e5c31447d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:16.115 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': 'A27233BD-51BD-4F89-A24E-4B9ECDE03377', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15049), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:16.252 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:16.252 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:16.252 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:16.252 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:16.252 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': 'A27233BD-51BD-4F89-A24E-4B9ECDE03377', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112063), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 16), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 16), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:16.252 4076 3051113163240 MainThread DEBUG os_win._utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 4. Time left: undefined. Time elapsed: 1.342310905456543 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:42:16.278 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-cb85002f-1930-4e34-b440-ca01ef472a89 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:16.278 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-cb85002f-1930-4e34-b440-ca01ef472a89 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:16.319 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-cb85002f-1930-4e34-b440-ca01ef472a89 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:16.348 4076 3051055217304 MainThread INFO nova.compute.manager [req-cb85002f-1930-4e34-b440-ca01ef472a89 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:16.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:16.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:16.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:17.278 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '51E22F21-B57E-4B61-A4F5-F68B27C36599', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=17155), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:17.413 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:17.413 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:17.413 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:17.413 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:17.426 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '51E22F21-B57E-4B61-A4F5-F68B27C36599', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=113076), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 17), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 17), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:17.426 4076 3051113163240 MainThread DEBUG os_win._utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 3. Time left: undefined. Time elapsed: 2.5159053802490234 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:42:17.426 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-512a9f06-3868-493a-93f5-e4324aef5a7d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:17.426 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-512a9f06-3868-493a-93f5-e4324aef5a7d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:17.485 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-512a9f06-3868-493a-93f5-e4324aef5a7d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:17.512 4076 3051055217304 MainThread INFO nova.compute.manager [req-512a9f06-3868-493a-93f5-e4324aef5a7d - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:17.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:18.441 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '10A99AF9-F172-4969-84D7-DB5631155A6D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14943), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:18.563 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:18.563 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:18.563 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:18.563 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:18.585 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '10A99AF9-F172-4969-84D7-DB5631155A6D', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=112356), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 18), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 18), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:18.585 4076 3051113163240 MainThread DEBUG os_win._utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 2. Time left: undefined. Time elapsed: 3.674945592880249 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:42:18.597 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-538372ea-f12f-4c54-95e4-e3e4b670dc52 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:18.597 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-538372ea-f12f-4c54-95e4-e3e4b670dc52 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:18.627 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-538372ea-f12f-4c54-95e4-e3e4b670dc52 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:18.651 4076 3051055217304 MainThread INFO nova.compute.manager [req-538372ea-f12f-4c54-95e4-e3e4b670dc52 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:19.629 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '08F1E01B-DD70-4435-A1D0-3268DB6E3292', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15286), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:19.740 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:19.740 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:19.752 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:19.752 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:19.752 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:42:19.752 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:42:19.766 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:19.766 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:19.766 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:19.766 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:19.766 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:42:19.786 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '08F1E01B-DD70-4435-A1D0-3268DB6E3292', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111289), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 19), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 19), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:19.786 4076 3051113163240 MainThread DEBUG os_win._utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Got expected exception WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. while calling function os_win.utils.compute.vmutils.VMUtils.set_vm_state. Retries left: 1. Time left: undefined. Time elapsed: 4.876075744628906 Retrying in 1 seconds. inner c:\openstack\build\os-win\os_win\_utils.py:195 2021-09-10 17:42:19.786 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-1b158f52-489f-422a-8f0d-b586a51a738c - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:19.786 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-1b158f52-489f-422a-8f0d-b586a51a738c - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:19.828 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-1b158f52-489f-422a-8f0d-b586a51a738c - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:19.856 4076 3051055217304 MainThread INFO nova.compute.manager [req-1b158f52-489f-422a-8f0d-b586a51a738c - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:20.001 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2193MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:42:20.018 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2423d019-a7f7-433a-9baf-c071866eaccd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 13 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:42:20.310 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=13824MB phys_disk=79GB used_disk=13GB total_vcpus=6 used_vcpus=13 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:42:20.380 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:42:20.392 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:42:20.392 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:42:20.420 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:42:20.469 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:42:20.469 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:42:20.488 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:42:20.675 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:42:20.705 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:42:20.705 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:42:20.705 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.687s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:20.806 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '8EC081A9-177D-4668-9FA8-A4A6C074A458', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=14554), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:20.925 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:20.925 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:20.944 4076 3051055217304 MainThread DEBUG nova.virt.driver [-] Emitting event Stopped> emit_event c:\openstack\build\nova\nova\virt\driver.py:1609 2021-09-10 17:42:20.944 4076 3051055217304 MainThread INFO nova.compute.manager [-] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] VM Stopped (Lifecycle Event) 2021-09-10 17:42:20.944 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:20.949 4076 3051113163240 MainThread ERROR os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job failed: {'RawErrors': (('124\'instance-00000095\' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF12030Microsoft-Windows-Hyper-V-Worker50', '124Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF10243122Microsoft-Windows-Hyper-V-Worker50', '124\'instance-00000095\' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)instance-000000951CD2E99D-C169-486C-891A-4C5A230B65FF%%21479424140x8007000E3050Microsoft-Windows-Hyper-V-Worker50'), 0), 'InstanceID': '8EC081A9-177D-4668-9FA8-A4A6C074A458', 'Description': 'Initializing and Starting Virtual Machine', 'ElementName': 'Starting', 'JobStatus': 'Job is completed with error', 'ElapsedTime': datetime.timedelta(microseconds=111824), 'Cancellable': True, 'JobType': 12, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 10, 'JobState': 10, 'StatusDescriptions': ('Job is completed', 'Job is completed with error'), 'OperationalStatus': (17, 6), 'TimeSubmitted': datetime.datetime(2021, 9, 10, 17, 42, 20), 'UntilTime': None, 'TimeOfLastStateChange': datetime.datetime(2021, 9, 10, 17, 42, 20), 'DetailedStatus': None, 'LocalOrUtcTime': 2, 'ErrorCode': 32778, 'ErrorDescription': "'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\nNot enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)\n\n'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF)", 'ErrorSummaryDescription': "'instance-00000095' failed to start.\n\nNot enough memory in the system to start the virtual machine instance-00000095.\n\nCould not initialize memory: Ran out of memory (0x8007000E)."}. 2021-09-10 17:42:20.949 4076 3051113163240 MainThread ERROR nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Failed to change vm state of instance-00000095 to 2: os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. 2021-09-10 17:42:20.949 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Got request to destroy instance 2021-09-10 17:42:20.949 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:20.949 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-275e853d-f777-4cb2-9eb4-da4aa9b2fcb4 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Checking state _get_power_state c:\openstack\build\nova\nova\compute\manager.py:1564 2021-09-10 17:42:20.949 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [req-275e853d-f777-4cb2-9eb4-da4aa9b2fcb4 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:42:21.015 4076 3051055217304 MainThread DEBUG nova.compute.manager [req-275e853d-f777-4cb2-9eb4-da4aa9b2fcb4 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Synchronizing instance power state after lifecycle event "Stopped"; current vm_state: building, current task_state: spawning, current DB power_state: 0, VM power_state: 4 handle_lifecycle_event c:\openstack\build\nova\nova\compute\manager.py:1294 2021-09-10 17:42:21.034 4076 3051055217304 MainThread INFO nova.compute.manager [req-275e853d-f777-4cb2-9eb4-da4aa9b2fcb4 - - - - -] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During sync_power_state the instance has a pending task (spawning). Skip. 2021-09-10 17:42:22.746 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Power off instance power_off c:\openstack\build\nova\nova\virt\hyperv\vmops.py:832 2021-09-10 17:42:22.746 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:22.746 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:22.762 4076 3051113163240 MainThread DEBUG os_win.utils.compute.vmutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Successfully changed vm state of instance-00000095 to 3 set_vm_state c:\openstack\build\os-win\os_win\utils\compute\vmutils.py:698 2021-09-10 17:42:22.762 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Successfully changed state of VM instance-00000095 to: 3 _set_vm_state c:\openstack\build\nova\nova\virt\hyperv\vmops.py:878 2021-09-10 17:42:22.814 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Waiting for WMI job: {'InstanceID': '0F331419-9048-4E82-B72D-7AD0FA5D9A27', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is running', 'ElapsedTime': datetime.timedelta(microseconds=15006), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 0}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:94 2021-09-10 17:42:22.933 4076 3051113163240 MainThread DEBUG os_win.utils.jobutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] WMI job succeeded: {'InstanceID': '0F331419-9048-4E82-B72D-7AD0FA5D9A27', 'Description': 'Destroying Virtual Machine', 'ElementName': 'Destroying', 'JobStatus': 'Job is completed', 'ElapsedTime': datetime.timedelta(microseconds=29098), 'Cancellable': False, 'JobType': 3, 'Owner': 'CBCI-808474-1\\administrator', 'PercentComplete': 100}. _wait_for_job c:\openstack\build\os-win\os_win\utils\jobutils.py:124 2021-09-10 17:42:22.933 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:22.933 4076 3051113163240 MainThread DEBUG os_win.utils.pathutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Removing directory: C:\openstack\instances\instance-00000095 check_remove_dir c:\openstack\build\os-win\os_win\utils\pathutils.py:151 2021-09-10 17:42:22.933 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:22.933 4076 3051113163240 MainThread ERROR nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Instance failed to spawn: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Traceback (most recent call last): 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(*args, **kwargs) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] admin_password, network_info, block_device_info) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(self, *args, **kwds) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] should_plug_vifs=False) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'req_state': req_state}) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] time.sleep(sleep_time) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return f(*args, **kwargs) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return self._wait_for_job(job_path) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] error_desc=err_desc) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During handling of the above exception, another exception occurred: 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Traceback (most recent call last): 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2638, in _build_resources 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] yield resources 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] accel_info=accel_info) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(*args, **kwargs) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] admin_password, network_info, block_device_info) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(self, *args, **kwds) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] should_plug_vifs=False) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'req_state': req_state}) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] time.sleep(sleep_time) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return f(*args, **kwargs) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return self._wait_for_job(job_path) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] error_desc=err_desc) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:22.933 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:22.939 4076 3051113163240 MainThread INFO nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Terminating instance 2021-09-10 17:42:22.939 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Start destroying the instance on the hypervisor. _shutdown_instance c:\openstack\build\nova\nova\compute\manager.py:2895 2021-09-10 17:42:22.939 4076 3051113163240 MainThread INFO nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Got request to destroy instance 2021-09-10 17:42:22.951 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Instance not found destroy c:\openstack\build\nova\nova\virt\hyperv\vmops.py:744 2021-09-10 17:42:22.951 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" acquired by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:22.951 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "instance-00000095" released by "nova.virt.hyperv.serialconsoleops.instance_synchronized..wrapper..inner" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:22.951 4076 3051113163240 MainThread INFO nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Took 0.01 seconds to destroy the instance on the hypervisor. 2021-09-10 17:42:22.951 4076 3051113163240 MainThread DEBUG nova.compute.claims [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Aborting claim: abort c:\openstack\build\nova\nova\compute\claims.py:84 2021-09-10 17:42:22.951 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:42:23.044 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:42:23.064 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.hostops [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:42:23.265 4076 3051113163240 MainThread DEBUG nova.compute.provider_tree [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG nova.scheduler.client.report [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.abort_instance_claim" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:23.278 4076 3051113163240 MainThread ERROR nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Failed to build and run instance: nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Traceback (most recent call last): 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(*args, **kwargs) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] admin_password, network_info, block_device_info) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(self, *args, **kwds) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] should_plug_vifs=False) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'req_state': req_state}) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] time.sleep(sleep_time) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return f(*args, **kwargs) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return self._wait_for_job(job_path) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] error_desc=err_desc) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] os_win.exceptions.WMIJobFailed: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] During handling of the above exception, another exception occurred: 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Traceback (most recent call last): 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\compute\manager.py", line 2402, in _build_and_run_instance 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] accel_info=accel_info) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 67, in wrapper 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise exc.with_traceback(exc_info[2]) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 53, in wrapper 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(*args, **kwargs) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\driver.py", line 163, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] admin_password, network_info, block_device_info) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 81, in wrapper 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return function(self, *args, **kwds) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 304, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.destroy(instance, network_info, block_device_info) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 301, in spawn 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] should_plug_vifs=False) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 868, in power_on 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._set_vm_state(instance, os_win_const.HYPERV_VM_STATE_ENABLED) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 884, in _set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'req_state': req_state}) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\nova\nova\virt\hyperv\vmops.py", line 874, in _set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._vmutils.set_vm_state(instance_name, req_state) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 196, in inner 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] time.sleep(sleep_time) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 227, in __exit__ 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self.force_reraise() 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\python37\lib\site-packages\oslo_utils\excutils.py", line 200, in force_reraise 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] raise self.value 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\_utils.py", line 149, in inner 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return f(*args, **kwargs) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\compute\vmutils.py", line 695, in set_vm_state 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] self._jobutils.check_ret_val(ret_val, job_path, [0, 32775]) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 73, in check_ret_val 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] return self._wait_for_job(job_path) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] File "c:\openstack\build\os-win\os_win\utils\jobutils.py", line 117, in _wait_for_job 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] error_desc=err_desc) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] nova.exception.NovaException: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. 2021-09-10 17:42:23.278 4076 3051113163240 MainThread TRACE nova.compute.manager [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG nova.compute.utils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error descriptio notify_about_instance_usage c:\openstack\build\nova\nova\compute\utils.py:426 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Build of instance 2423d019-a7f7-433a-9baf-c071866eaccd was re-scheduled: WMI job failed with status 10. Error summary description: 'instance-00000095' failed to start. Not enough memory in the system to start the virtual machine instance-00000095. Could not initialize memory: Ran out of memory (0x8007000E).. Error description: 'instance-00000095' failed to start. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Not enough memory in the system to start the virtual machine instance-00000095 with ram size 1024 megabytes. (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) 'instance-00000095' could not initialize memory: Ran out of memory (0x8007000E). (Virtual machine ID 1CD2E99D-C169-486C-891A-4C5A230B65FF) Error code: 32778. _do_build_and_run_instance c:\openstack\build\nova\nova\compute\manager.py:2243 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Unplugging VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2750 2021-09-10 17:42:23.278 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Unplugged VIFs for instance _cleanup_allocated_networks c:\openstack\build\nova\nova\compute\manager.py:2786 2021-09-10 17:42:23.300 4076 3051113163240 MainThread DEBUG nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Deallocating network for instance _deallocate_network c:\openstack\build\nova\nova\compute\manager.py:2057 2021-09-10 17:42:23.300 4076 3051113163240 MainThread DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] deallocate_for_instance() deallocate_for_instance c:\openstack\build\nova\nova\network\neutron.py:1672 2021-09-10 17:42:23.765 4076 3051113163240 MainThread DEBUG nova.network.neutron [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Updating instance_info_cache with network_info: [] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:42:23.795 4076 3051113163240 MainThread INFO nova.compute.manager [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] [instance: 2423d019-a7f7-433a-9baf-c071866eaccd] Took 0.50 seconds to deallocate network for instance. 2021-09-10 17:42:23.956 4076 3051113163240 MainThread INFO nova.scheduler.client.report [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Deleted allocations for instance 2423d019-a7f7-433a-9baf-c071866eaccd 2021-09-10 17:42:23.989 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [req-a1994e4e-8c3c-4dcf-a5b5-f9c1728dc4e5 10226e0397b7435499d0680801742547 3efd33abd6ee4c68bcbc61a4464c4134 - default default] Lock "2423d019-a7f7-433a-9baf-c071866eaccd" released by "nova.compute.manager.ComputeManager.build_and_run_instance.._locked_do_build_and_run_instance" :: held 18.406s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:42:37.753 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:37.753 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:42:49.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:42:49.745 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:42:49.761 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:43:09.771 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:11.752 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:11.752 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:43:11.815 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:43:11.815 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:43:12.315 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updating instance_info_cache with network_info: [{"id": "799c7188-eef9-4b77-8d75-2cfc73d1a87e", "address": "fa:16:3e:aa:8d:2c", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.116", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap799c7188-ee", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:43:12.332 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-6c4f2594-1a7c-49a9-8178-9f3072319512" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:43:12.332 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:43:12.740 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:12.740 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:43:18.708 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:18.743 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:18.755 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:20.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:20.760 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:43:20.760 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:43:20.760 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:43:20.760 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:43:20.777 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:43:20.996 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2110MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:43:20.996 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:43:21.135 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:43:21.155 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:43:21.185 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:43:21.385 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:43:21.386 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:43:21.386 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:43:21.386 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:43:22.380 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:22.380 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:43:28.729 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:11.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:11.738 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:44:11.794 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:44:11.794 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:44:12.295 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updating instance_info_cache with network_info: [{"id": "0110d67d-01ad-4b98-9caf-80b01bc70bfb", "address": "fa:16:3e:fa:46:99", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.113", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0110d67d-01", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:44:12.322 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:44:12.322 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:44:12.322 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:12.744 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:12.744 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:44:18.722 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:19.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:21.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:21.738 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:44:21.738 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:44:21.738 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:44:21.738 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:44:21.794 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:44:22.019 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1952MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:44:22.025 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:44:22.285 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.285 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.302 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:44:22.309 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:44:22.309 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:44:22.328 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:44:22.338 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:44:22.546 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:44:22.559 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:44:22.559 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:44:22.559 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.532s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:44:23.561 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:23.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:44:25.962 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:11.768 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:11.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:45:11.930 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:45:11.930 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:45:12.674 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updating instance_info_cache with network_info: [{"id": "5beac292-790e-4830-8f59-e411516f08cc", "address": "fa:16:3e:ea:47:a7", "network": {"id": "7434a41e-b78f-4cb9-8ffe-b2514d097201", "bridge": null, "label": "tempest-LiveAutoBlockMigrationV225Test-1499875764-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "91237ebda1b54d2dbc761e48fdd93516", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5beac292-79", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:45:12.689 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-9107836a-481d-4cb5-9c7d-6432f4cd5309" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:45:12.689 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:45:12.743 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:13.758 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:13.758 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:45:20.711 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:20.742 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:21.736 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:23.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:23.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:23.745 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:45:23.745 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:45:23.745 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:45:23.745 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:45:23.784 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:45:24.000 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2006MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:45:24.000 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:45:24.127 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.148 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.148 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:45:24.150 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:45:24.170 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:45:24.379 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:45:24.383 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:45:24.383 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:45:24.383 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:45:26.382 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:45:29.710 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:12.743 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:12.743 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:46:12.743 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Rebuilding the list of instances to heal _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9342 2021-09-10 17:46:12.828 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:46:12.828 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:46:12.828 4076 3051049642296 GreenThread-2 DEBUG nova.objects.instance [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lazy-loading 'info_cache' on Instance uuid 7ed088c4-2147-476f-b179-befc33eb129d obj_load_attr c:\openstack\build\nova\nova\objects\instance.py:1101 2021-09-10 17:46:13.345 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updating instance_info_cache with network_info: [{"id": "c0c8e7e4-c81a-47f5-8d82-27a129730b8e", "address": "fa:16:3e:3d:cd:5a", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.67", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapc0c8e7e4-c8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:46:13.368 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-7ed088c4-2147-476f-b179-befc33eb129d" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:46:13.368 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:46:13.741 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:13.741 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:46:14.737 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:21.725 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:21.725 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_shelved_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:21.756 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:21.759 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:23.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:23.751 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:46:23.751 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:46:23.751 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:46:23.751 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:46:23.766 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:46:23.975 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2008MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:46:23.975 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.107 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.123 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:46:24.123 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:46:24.123 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:46:24.123 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:46:24.134 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:46:24.360 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:46:24.384 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:46:24.384 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:46:24.384 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.407s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:46:26.381 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:26.740 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:46:28.943 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:12.788 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:12.788 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:47:12.943 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:47:12.943 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:47:13.396 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updating instance_info_cache with network_info: [{"id": "5cc5f59c-ef6e-4ba0-b041-e283e7d8339a", "address": "fa:16:3e:3e:97:15", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap5cc5f59c-ef", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:47:13.427 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-e81b57e5-a1e7-417d-9607-9f17f3aeafdd" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:47:13.427 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:47:15.739 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:15.739 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:47:16.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:17.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:21.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:21.747 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:23.708 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:23.749 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:23.765 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:47:23.765 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:47:23.765 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:47:23.765 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:47:23.780 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:47:23.988 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=2005MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:47:24.005 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:47:24.119 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:47:24.155 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:47:24.177 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:47:24.177 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:47:24.203 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:47:24.242 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:47:24.242 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:47:24.265 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:47:24.468 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:47:24.485 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:47:24.485 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:47:24.485 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.484s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:47:26.456 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:28.736 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:34.711 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:39.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_incomplete_migrations run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:39.754 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances with incomplete migration _cleanup_incomplete_migrations c:\openstack\build\nova\nova\compute\manager.py:10640 2021-09-10 17:47:53.749 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._run_pending_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:47:53.749 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Cleaning up deleted instances _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10602 2021-09-10 17:47:53.749 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] There are 0 instances to clean _run_pending_deletes c:\openstack\build\nova\nova\compute\manager.py:10611 2021-09-10 17:48:14.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:14.738 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:48:14.816 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:48:14.816 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:48:15.304 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updating instance_info_cache with network_info: [{"id": "298cc953-6875-4186-9b30-11fa49791716", "address": "fa:16:3e:fa:ac:f1", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.136", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap298cc953-68", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:48:15.304 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-f830cec8-b836-4470-83fd-f48ca352a795" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:48:15.304 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: f830cec8-b836-4470-83fd-f48ca352a795] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:48:15.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:15.754 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:48:17.733 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:21.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:23.701 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:23.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:23.754 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:23.769 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:48:23.769 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:48:23.769 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:48:23.769 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:48:23.769 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:48:23.981 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1928MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:48:23.981 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:48:24.113 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:48:24.144 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:48:24.350 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:48:24.368 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:48:24.368 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:48:24.368 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:48:26.354 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:29.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:48:29.935 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:03.956 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_power_states run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 7ed088c4-2147-476f-b179-befc33eb129d _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid e81b57e5-a1e7-417d-9607-9f17f3aeafdd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid f830cec8-b836-4470-83fd-f48ca352a795 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid a995bd38-7c42-44fb-b4c9-d8819357d630 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid ca35d63c-0cc3-4334-9b0d-fc30961067d5 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 6c4f2594-1a7c-49a9-8178-9f3072319512 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Triggering sync for uuid 9107836a-481d-4cb5-9c7d-6432f4cd5309 _sync_power_states c:\openstack\build\nova\nova\compute\manager.py:9748 2021-09-10 17:49:04.020 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113161480 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 17abaf80-43c7-4c09-b4a2-c12636cd0cc3] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.020 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113163240 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.020 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113163064 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: e81b57e5-a1e7-417d-9607-9f17f3aeafdd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.020 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113164472 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: ca35d63c-0cc3-4334-9b0d-fc30961067d5] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.020 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113161656 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 6c4f2594-1a7c-49a9-8178-9f3072319512] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.020 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.020 4076 3051113164120 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051113160776 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: f830cec8-b836-4470-83fd-f48ca352a795] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051113163944 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051093170664 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 9107836a-481d-4cb5-9c7d-6432f4cd5309] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051113162712 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051093169960 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.039 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:04.039 4076 3051055217304 MainThread DEBUG nova.virt.hyperv.vmops [-] [instance: 7ed088c4-2147-476f-b179-befc33eb129d] get_info called for instance get_info c:\openstack\build\nova\nova\virt\hyperv\vmops.py:121 2021-09-10 17:49:04.243 4076 3051113164472 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "ca35d63c-0cc3-4334-9b0d-fc30961067d5" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.219s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.243 4076 3051113160776 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "f830cec8-b836-4470-83fd-f48ca352a795" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.243 4076 3051113163944 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.243 4076 3051055217304 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "7ed088c4-2147-476f-b179-befc33eb129d" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.243 4076 3051113162712 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.203s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.362 4076 3051113164120 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "a995bd38-7c42-44fb-b4c9-d8819357d630" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.344s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.362 4076 3051093170664 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "9107836a-481d-4cb5-9c7d-6432f4cd5309" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.328s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.393 4076 3051113161656 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "6c4f2594-1a7c-49a9-8178-9f3072319512" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.399 4076 3051113163240 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.399 4076 3051093169960 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.359s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.399 4076 3051113161480 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "17abaf80-43c7-4c09-b4a2-c12636cd0cc3" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:04.399 4076 3051113163064 MainThread DEBUG oslo_concurrency.lockutils [-] Lock "e81b57e5-a1e7-417d-9607-9f17f3aeafdd" released by "nova.compute.manager.ComputeManager._sync_power_states.._sync..query_driver_power_state_and_sync" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:15.799 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:15.799 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:49:15.872 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:49:15.872 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:49:16.350 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updating instance_info_cache with network_info: [{"id": "df65fbf2-3d95-4273-adfa-5d65cf85d7c1", "address": "fa:16:3e:ef:f1:83", "network": {"id": "7269c681-e442-4edc-a7f7-eb367d70620b", "bridge": null, "label": "tempest-ServersWithSpecificFlavorTestJSON-568615719-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "145f8177efc74f5e9be31c5f74e69d0a", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tapdf65fbf2-3d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:49:16.379 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-a995bd38-7c42-44fb-b4c9-d8819357d630" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:49:16.379 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: a995bd38-7c42-44fb-b4c9-d8819357d630] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:49:17.731 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:17.731 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:49:19.741 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:21.736 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:23.736 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:24.716 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:24.752 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:24.768 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:24.768 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:24.768 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:49:24.768 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:49:24.809 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:49:25.031 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1989MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:49:25.031 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:49:25.259 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:49:25.277 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:49:25.473 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:49:25.502 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:49:25.502 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:49:25.502 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.469s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:49:26.493 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:31.736 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:49:36.705 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:15.746 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:15.746 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:50:15.846 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:50:15.846 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:50:16.355 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updating instance_info_cache with network_info: [{"id": "54260358-c3ac-4d72-8f2e-1153615d8bdb", "address": "fa:16:3e:be:72:b4", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.234", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap54260358-c3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:50:16.369 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:50:16.369 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:50:18.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:18.735 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:50:21.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:21.738 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:24.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:24.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:24.751 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:50:24.751 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:50:24.751 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:50:24.756 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:50:24.756 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:50:24.973 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1975MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:50:24.973 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:50:25.102 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:50:25.110 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:50:25.136 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:50:25.355 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:50:25.361 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:50:25.361 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:50:25.361 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.391s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:50:26.360 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:26.699 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:31.735 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:50:34.938 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:15.750 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:15.750 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:51:15.814 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:51:15.814 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:51:16.306 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updating instance_info_cache with network_info: [{"id": "0519e0a6-143b-4dde-87d3-52c308238e7f", "address": "fa:16:3e:f1:02:24", "network": {"id": "f5731852-5afb-46d1-b2dd-91163c55904a", "bridge": null, "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.25", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.2"}}], "meta": {"injected": false, "tenant_id": "b066f936ad564c189113f73748329e33", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap0519e0a6-14", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:51:16.317 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:51:16.317 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:51:20.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:20.730 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:51:21.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:22.729 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:24.749 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:25.734 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:26.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:26.745 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:51:26.745 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:51:26.745 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:51:26.765 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:51:26.801 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:51:27.031 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1931MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:51:27.031 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:51:27.157 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:51:27.173 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:51:27.390 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:51:27.408 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:51:27.408 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:51:27.408 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.375s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:51:29.359 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:31.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:51:40.718 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._sync_scheduler_instance_info run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:16.733 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._heal_instance_info_cache run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:16.733 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Starting heal instance info cache _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9338 2021-09-10 17:52:16.853 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Acquired lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:266 2021-09-10 17:52:16.853 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Forcefully refreshing network info cache for instance _get_instance_nw_info c:\openstack\build\nova\nova\network\neutron.py:1871 2021-09-10 17:52:17.295 4076 3051049642296 GreenThread-2 DEBUG nova.network.neutron [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updating instance_info_cache with network_info: [{"id": "02ca13c3-f2a9-4341-b400-dd8762c5f142", "address": "fa:16:3e:01:92:31", "network": {"id": "03c1eefd-030f-414f-b7f8-81ed611f877f", "bridge": null, "label": "tempest-LiveMigrationTest-812207682-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "fa8236f1477243dfafd0682b608fc659", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false}, "devname": "tap02ca13c3-f2", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] update_instance_cache_with_nw_info c:\openstack\build\nova\nova\network\neutron.py:118 2021-09-10 17:52:17.326 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Releasing lock "refresh_cache-2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4" lock c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:282 2021-09-10 17:52:17.326 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] [instance: 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4] Updated the network info_cache for instance _heal_instance_info_cache c:\openstack\build\nova\nova\compute\manager.py:9410 2021-09-10 17:52:21.732 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_unconfirmed_resizes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:21.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:22.748 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:22.748 4076 3051049642296 GreenThread-2 DEBUG nova.compute.manager [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes c:\openstack\build\nova\nova\compute\manager.py:9955 2021-09-10 17:52:23.733 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rescued_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:24.745 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_volume_usage run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:25.730 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:27.729 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:27.733 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:52:27.733 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:52:27.733 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Auditing locally available compute resources for n-h1-808474-1 (node: n-h1-808474-1) update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:878 2021-09-10 17:52:27.733 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:52:27.785 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:52:28.013 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Hypervisor/Node resource view: name=n-h1-808474-1 free_ram=1931MB free_disk=50GB free_vcpus=6 pci_devices=[] _report_hypervisor_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1026 2021-09-10 17:52:28.013 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:359 2021-09-10 17:52:28.151 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 7ed088c4-2147-476f-b179-befc33eb129d actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.151 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance f830cec8-b836-4470-83fd-f48ca352a795 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.151 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance a995bd38-7c42-44fb-b4c9-d8819357d630 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.151 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4fc4b80d-f5c5-4090-9ab3-9c4ca4663f34 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance e81b57e5-a1e7-417d-9607-9f17f3aeafdd actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 5d8dfa6d-ba29-4d1b-a73c-a2e159c62d7c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 2b09c2b7-7d01-4ecb-b8d5-3e937fcf9ab4 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance ca35d63c-0cc3-4334-9b0d-fc30961067d5 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 6c4f2594-1a7c-49a9-8178-9f3072319512 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 17abaf80-43c7-4c09-b4a2-c12636cd0cc3 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 4f7cc1cd-7a8a-4a2c-acee-fd6f1ed632fd actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 1024, 'VCPU': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Instance 9107836a-481d-4cb5-9c7d-6432f4cd5309 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 1024, 'DISK_GB': 1}}. _remove_deleted_instances_allocations c:\openstack\build\nova\nova\compute\resource_tracker.py:1532 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Total usable vcpus: 6, total allocated vcpus: 12 _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1042 2021-09-10 17:52:28.164 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Final resource view: name=n-h1-808474-1 phys_ram=16383MB used_ram=12800MB phys_disk=79GB used_disk=12GB total_vcpus=6 used_vcpus=12 pci_stats=[] _report_final_resource_view c:\openstack\build\nova\nova\compute\resource_tracker.py:1064 2021-09-10 17:52:28.183 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing inventories for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:757 2021-09-10 17:52:28.202 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating ProviderTree inventory for provider 6729133d-6e37-4062-bd58-46fc02d92e9d from _refresh_and_get_inventory using data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} _refresh_and_get_inventory c:\openstack\build\nova\nova\scheduler\client\report.py:723 2021-09-10 17:52:28.202 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Updating inventory in ProviderTree for provider 6729133d-6e37-4062-bd58-46fc02d92e9d with inventory: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:176 2021-09-10 17:52:28.239 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing aggregate associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, aggregates: None _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:766 2021-09-10 17:52:28.282 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Refreshing trait associations for resource provider 6729133d-6e37-4062-bd58-46fc02d92e9d, traits: COMPUTE_DEVICE_TAGGING,COMPUTE_SECURITY_UEFI_SECURE_BOOT,COMPUTE_IMAGE_TYPE_VHDX,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_NODE,COMPUTE_IMAGE_TYPE_VHD _refresh_associations c:\openstack\build\nova\nova\scheduler\client\report.py:778 2021-09-10 17:52:28.282 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] get_available_resource called get_available_resource c:\openstack\build\nova\nova\virt\hyperv\hostops.py:147 2021-09-10 17:52:28.282 4076 3051049642296 GreenThread-2 DEBUG nova.virt.hyperv.hostops [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Windows version: 10000 _get_hypervisor_version c:\openstack\build\nova\nova\virt\hyperv\hostops.py:107 2021-09-10 17:52:28.502 4076 3051049642296 GreenThread-2 DEBUG nova.compute.provider_tree [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed in ProviderTree for provider: 6729133d-6e37-4062-bd58-46fc02d92e9d update_inventory c:\openstack\build\nova\nova\compute\provider_tree.py:180 2021-09-10 17:52:28.502 4076 3051049642296 GreenThread-2 DEBUG nova.scheduler.client.report [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Inventory has not changed for provider 6729133d-6e37-4062-bd58-46fc02d92e9d based on inventory data: {'VCPU': {'total': 6, 'reserved': 0, 'min_unit': 1, 'max_unit': 6, 'step_size': 1, 'allocation_ratio': 16.0}, 'MEMORY_MB': {'total': 16383, 'reserved': 512, 'min_unit': 1, 'max_unit': 16383, 'step_size': 1, 'allocation_ratio': 1.5}, 'DISK_GB': {'total': 79, 'reserved': 0, 'min_unit': 1, 'max_unit': 79, 'step_size': 1, 'allocation_ratio': 1.0}} set_inventory_for_provider c:\openstack\build\nova\nova\scheduler\client\report.py:894 2021-09-10 17:52:28.527 4076 3051049642296 GreenThread-2 DEBUG nova.compute.resource_tracker [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Compute_service record updated for n-h1-808474-1:n-h1-808474-1 _update_available_resource c:\openstack\build\nova\nova\compute\resource_tracker.py:977 2021-09-10 17:52:28.527 4076 3051049642296 GreenThread-2 DEBUG oslo_concurrency.lockutils [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.516s inner c:\python37\lib\site-packages\oslo_concurrency\lockutils.py:371 2021-09-10 17:52:30.511 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._check_instance_build_time run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:33.729 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211 2021-09-10 17:52:34.936 4076 3051049642296 GreenThread-2 DEBUG oslo_service.periodic_task [req-08ee348a-cfa0-4deb-9e90-cca1e41c509d - - - - -] Running periodic task ComputeManager._cleanup_running_deleted_instances run_periodic_tasks c:\python37\lib\site-packages\oslo_service\periodic_task.py:211